Jul 23 03:55:17.405063 ip-10-0-138-244 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 03:55:17.853522 ip-10-0-138-244 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:17.853522 ip-10-0-138-244 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 03:55:17.853522 ip-10-0-138-244 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:17.853522 ip-10-0-138-244 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 03:55:17.853522 ip-10-0-138-244 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 03:55:17.855638 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.855553    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 03:55:17.863017 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863000    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.863017 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863016    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.863017 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863021    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863025    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863030    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863033    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863037    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863041    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863046    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863049    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863053    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863057    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863071    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863075    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863079    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863083    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863087    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863091    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863095    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863099    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863103    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863107    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.863213 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863111    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863115    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863119    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863123    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863127    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863131    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863135    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863140    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863144    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863148    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863154    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863158    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863162    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863167    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863173    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863177    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863182    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863186    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863191    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.864028 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863195    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863199    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863203    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863207    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863214    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863219    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863224    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863229    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863234    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863238    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863245    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863252    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863256    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863261    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863265    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863269    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863273    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863277    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863281    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.864828 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863286    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863290    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863294    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863298    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863302    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863306    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863319    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863324    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863330    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863337    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863343    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863348    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863356    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863361    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863366    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863370    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863374    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863378    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863382    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.865579 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863386    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863391    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863394    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863399    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863403    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863408    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863412    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863416    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863420    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863425    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863428    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863433    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863438    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863442    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863446    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863450    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863454    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863459    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863463    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.863467    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.866329 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864596    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864605    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864611    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864616    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864620    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864625    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864629    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864633    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864637    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864641    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864646    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864651    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864655    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864659    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864663    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864667    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864672    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864677    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864681    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.866922 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864686    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864690    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864696    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864701    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864706    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864710    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864715    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864719    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864723    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864730    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864736    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864741    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864745    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864750    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864756    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864760    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864764    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864769    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864775    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864780    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.867379 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864784    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864789    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864793    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864797    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864819    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864823    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864827    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864831    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864835    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864839    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864844    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864848    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864852    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864857    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864861    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864866    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864871    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864875    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864879    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.867908 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864883    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864887    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864891    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864895    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864899    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864903    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864908    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864913    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864917    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864922    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864926    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864931    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864935    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864943    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864949    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864955    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864960    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864964    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864969    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.868366 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864973    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864977    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864981    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864986    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864990    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864994    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.864998    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865002    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865006    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865011    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865015    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865019    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865023    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865028    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865032    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865036    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865040    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865044    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865048    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865053    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.868840 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865056    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.865060    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866180    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866194    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866199    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866205    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866209    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866215    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866219    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866223    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866227    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866231    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866235    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866240    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866249    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866253    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866257    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866262    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866267    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866271    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.869334 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866275    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866279    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866283    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866287    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866292    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866296    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866305    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866309    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866313    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866318    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866322    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866327    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866331    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866335    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866340    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866346    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866351    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866355    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866360    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866369    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.869835 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866373    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866377    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866381    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866385    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866390    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866394    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866398    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866402    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866406    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866410    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866414    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866423    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866427    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866431    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866435    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866439    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866443    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866448    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866452    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.870401 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866459    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866465    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866471    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866476    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866489    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866496    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866503    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866511    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866518    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866526    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866536    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866545    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866552    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866560    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866568    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866577    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866590    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866598    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866605    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.870884 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866613    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866619    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866627    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866635    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866642    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866649    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866657    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866665    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866675    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866687    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866694    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866701    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866708    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866715    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866721    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866730    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866737    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866754    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866760    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866767    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.871337 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866775    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866788    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.866796    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867172    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867198    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867206    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867211    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867216    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867220    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867224    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867229    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867232    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867236    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867239    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867243    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867246    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867249    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867252    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867255    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867258    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867261    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867266    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867269    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867272    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 03:55:17.871859 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867275    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867278    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867282    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867286    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867291    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867295    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867298    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867302    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867305    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867308    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867311    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867320    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867324    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867327    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867330    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867333    2582 flags.go:64] FLAG: --enable-server="true"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867336    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867342    2582 flags.go:64] FLAG: --event-burst="100"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867345    2582 flags.go:64] FLAG: --event-qps="50"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867349    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867351    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867354    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867358    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867361    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867364    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 03:55:17.872423 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867367    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867371    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867373    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867376    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867379    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867382    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867385    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867388    2582 flags.go:64] FLAG: --feature-gates=""
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867392    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867395    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867398    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867401    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867404    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867408    2582 flags.go:64] FLAG: --help="false"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867411    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-138-244.ec2.internal"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867414    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867417    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867420    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867425    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867430    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867433    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867436    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867440    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 03:55:17.873035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867443    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867446    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867449    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867452    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867455    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867458    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867461    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867464    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867467    2582 flags.go:64] FLAG: --lock-file=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867470    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867473    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867476    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867481    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867484    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867487    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867490    2582 flags.go:64] FLAG: --logging-format="text"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867493    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867496    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867499    2582 flags.go:64] FLAG: --manifest-url=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867502    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867506    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867509    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867514    2582 flags.go:64] FLAG: --max-pods="110"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867516    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867519    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 03:55:17.873610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867522    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867525    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867528    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867534    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867537    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867545    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867548    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867551    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867554    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867557    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867562    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867565    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867568    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867571    2582 flags.go:64] FLAG: --port="10250"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867574    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867577    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f8a2d1ea8c2eb68f"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867580    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867583    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867586    2582 flags.go:64] FLAG: --register-node="true"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867589    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867593    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867596    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867599    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867602    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867606    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 03:55:17.874222 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867609    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867612    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867615    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867617    2582 flags.go:64] FLAG: --runonce="false"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867620    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867623    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867626    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867629    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867632    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867635    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867638    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867643    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867646    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867649    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867651    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867654    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867657    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867660    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867663    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867669    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867672    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867675    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867679    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867682    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867685    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 03:55:17.874848 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867688    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867691    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867693    2582 flags.go:64] FLAG: --v="2"
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867698    2582 flags.go:64] FLAG: --version="false"
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867702    2582 flags.go:64] FLAG: --vmodule=""
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867712    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.867716    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867885    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867890    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867893    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867896    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867899    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867903    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867906    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867908    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867911    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867917    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867920    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867924    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867928    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867932    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.875497 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867935    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867939    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867942    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867945    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867948    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867951    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867954    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867956    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867959    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867962    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867964    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867967    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867970    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867973    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867975    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867978    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867982    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867986    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867989    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.876020 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867992    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867995    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.867998    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868001    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868003    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868006    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868009    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868011    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868015    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868018    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868022    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868024    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868027    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868030    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868032    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868035    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868038    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868040    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868043    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868046    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.876475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868049    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868052    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868056    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868059    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868062    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868065    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868067    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868070    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868073    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868075    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868078    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868081    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868084    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868086    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868089    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868092    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868094    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868097    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868099    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868102    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.876976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868105    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868108    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868112    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868115    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868117    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868120    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868123    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868126    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868128    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868131    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868134    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868136    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868139    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868142    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868145    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868147    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868150    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868152    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868155    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868157    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.877454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868160    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868163    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868165    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868168    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868172    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.868174    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.868969    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876209    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876224    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876243    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876248    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876301    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876306    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876310    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.877950 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876313    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876316    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876319    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876322    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876325    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876328    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876330    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876333    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876336    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876339    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876341    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876344    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876347    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876349    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876351    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876354    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876357    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876359    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876361    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.878290 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876366    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876369    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876372    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876375    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876378    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876381    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876384    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876387    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876390    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876392    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876395    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876397    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876400    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876402    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876405    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876407    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876410    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876413    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876415    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876418    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.878752 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876421    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876423    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876426    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876428    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876431    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876433    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876436    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876439    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876441    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876444    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876447    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876449    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876452    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876454    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876457    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876461    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876465    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876468    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876472    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876475    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.879254 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876478    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876481    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876484    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876487    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876489    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876492    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876494    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876497    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876500    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876502    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876505    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876507    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876510    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876513    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876515    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876517    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876520    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876523    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876525    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876528    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.879729 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876530    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876533    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876535    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876537    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876540    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876542    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876546    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876550    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876552    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876556    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876559    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876561    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876564    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876567    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876569    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876571    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.880216 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876574    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876579    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876692    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876696    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876699    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876702    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876705    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876709    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876712    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876715    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876717    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876720    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876722    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876725    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 03:55:17.880618 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876727    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876730    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876732    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876735    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876737    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876740    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876742    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876745    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876749    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876752    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876755    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876758    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876762    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876765    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876768    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876770    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876773    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876776    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876778    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876781    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 03:55:17.880976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876785    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876788    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876791    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876793    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876796    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876812    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876817    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876819    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876822    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876824    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876827    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876829    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876832    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876835    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876837    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876840    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876843    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876845    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876848    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 03:55:17.881454 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876851    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876853    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876856    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876859    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876861    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876864    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876867    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876870    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876873    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876875    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876878    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876881    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876883    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876886    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876888    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876891    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876893    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876896    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876898    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876901    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 03:55:17.881944 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876903    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876906    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876909    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876911    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876914    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876916    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876919    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876921    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876924    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876927    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876930    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876932    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876935    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876938    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876940    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876942    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876945    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876947    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876950    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876953    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 03:55:17.882418 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876956    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876958    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876960    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876963    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876966    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876968    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876971    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:17.876973    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.876978    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.877073    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.881083    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 03:55:17.882913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.882832    2582 server.go:1019] "Starting client certificate rotation"
Jul 23 03:55:17.883174 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.882933    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 03:55:17.883174 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.882967    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 03:55:17.885905 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.885892    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 03:55:17.887842 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.887829    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 03:55:17.902451 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.902435    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 23 03:55:17.905015 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.904996    2582 log.go:25] "Validated CRI v1 image API"
Jul 23 03:55:17.905771 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.905757    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 03:55:17.908713 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.908676    2582 fs.go:135] Filesystem UUIDs: map[2959d476-fb48-4a54-b5a2-b6e6672c2055:/dev/nvme0n1p3 56faf3da-2fab-4f72-bbf9-a779bc72640a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 23 03:55:17.908831 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.908715    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 03:55:17.913571 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.913466    2582 manager.go:216] Machine: {Timestamp:2026-07-23 03:55:17.913011621 +0000 UTC m=+0.374155286 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098406 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec201f2984b8ca2c8f3233278bc6761b SystemUUID:ec201f29-84b8-ca2c-8f32-33278bc6761b BootID:ff74a62b-03fd-4b61-936c-0b7c2f34c8d0 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:69:7b:45:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:69:7b:45:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:d6:99:c9:7a:f3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 03:55:17.913571 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.913560    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 03:55:17.913702 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.913654    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 03:55:17.914370 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.914351    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 03:55:17.915735 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.915713    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 03:55:17.915873 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.915738    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 03:55:17.916664 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.916650    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 03:55:17.916664 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.916664    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 03:55:17.916765 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.916671    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 03:55:17.917648 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.917637    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 03:55:17.919610 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.919601    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 03:55:17.924767 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924755    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 03:55:17.924908 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924898    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 03:55:17.924942 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924928    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 03:55:17.924942 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924941    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 03:55:17.925038 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924960    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 03:55:17.925038 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924973    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 23 03:55:17.925038 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.924984    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 03:55:17.925289 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.925278    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 03:55:17.925321 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.925295    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 03:55:17.927814 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.927783    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 03:55:17.930188 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.930164    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 03:55:17.930273 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.930202    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 03:55:17.931605 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931592    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931609    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931615    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931620    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931629    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931635    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931643    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931649    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 03:55:17.931652 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931656    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 03:55:17.931871 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931661    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 03:55:17.931871 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931670    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 03:55:17.931871 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.931675    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 03:55:17.932649 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.932635    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 03:55:17.932706 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.932655    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 03:55:17.935626 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.935612    2582 server.go:1268] "Started kubelet"
Jul 23 03:55:17.935738 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.935701    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 03:55:17.936391 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.936338    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 03:55:17.936480 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.936409    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 03:55:17.936645 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.936626    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 03:55:17.936824 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.936784    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 03:55:17.936901 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.936844    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 03:55:17.936988 ip-10-0-138-244 systemd[1]: Started Kubernetes Kubelet.
Jul 23 03:55:17.937522 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.937199    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 03:55:17.938456 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.938443    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 03:55:17.942432 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.942411    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 03:55:17.943239 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.942062    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-244.ec2.internal.18c4cef83e282735  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-244.ec2.internal,UID:ip-10-0-138-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-244.ec2.internal,},FirstTimestamp:2026-07-23 03:55:17.935589173 +0000 UTC m=+0.396732837,LastTimestamp:2026-07-23 03:55:17.935589173 +0000 UTC m=+0.396732837,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-244.ec2.internal,}"
Jul 23 03:55:17.947067 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.947051    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 03:55:17.947155 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.947073    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 03:55:17.947208 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.947166    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:17.948883 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.948865    2582 factory.go:55] Registering systemd factory
Jul 23 03:55:17.948969 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.948910    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 23 03:55:17.948969 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.948959    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 03:55:17.949065 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.948973    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 03:55:17.949065 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.948983    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 03:55:17.949170 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949129    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 03:55:17.949170 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949137    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 03:55:17.949254 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949221    2582 factory.go:153] Registering CRI-O factory
Jul 23 03:55:17.949254 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949234    2582 factory.go:223] Registration of the crio container factory successfully
Jul 23 03:55:17.949337 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949279    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 03:55:17.949337 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949300    2582 factory.go:103] Registering Raw factory
Jul 23 03:55:17.949337 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949324    2582 manager.go:1196] Started watching for new ooms in manager
Jul 23 03:55:17.949542 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.949519    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 03:55:17.949679 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.949650    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 03:55:17.949936 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.949922    2582 manager.go:349] Starting recovery of all containers
Jul 23 03:55:17.961449 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.961433    2582 manager.go:354] Recovery completed
Jul 23 03:55:17.962622 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.962600    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckbvf"
Jul 23 03:55:17.966487 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.966474    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:17.968276 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968258    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:17.968346 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968290    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:17.968346 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968300    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:17.968886 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968866    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 03:55:17.968944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968888    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 03:55:17.968944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968877    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:17.968944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.968906    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 03:55:17.969585 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.969572    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:17.969638 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.969600    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:17.969638 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.969615    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:17.969902 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.969887    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckbvf"
Jul 23 03:55:17.971214 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.971201    2582 policy_none.go:49] "None policy: Start"
Jul 23 03:55:17.971263 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.971222    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 03:55:17.971263 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.971233    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 03:55:17.973209 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:17.973199    2582 policy_none.go:47] "Start"
Jul 23 03:55:17.977084 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:17.976908    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-244.ec2.internal.18c4cef8401ae684  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-244.ec2.internal,UID:ip-10-0-138-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-244.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-244.ec2.internal,},FirstTimestamp:2026-07-23 03:55:17.968275076 +0000 UTC m=+0.429418744,LastTimestamp:2026-07-23 03:55:17.968275076 +0000 UTC m=+0.429418744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-244.ec2.internal,}"
Jul 23 03:55:18.015584 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.015569    2582 manager.go:337] "Starting Device Plugin manager"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.015605    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.015615    2582 server.go:87] "Starting device plugin registration server"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.015860    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.015872    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.015999    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.016068    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.016076    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.017855    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 03:55:18.029197 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.017897    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.044828 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.044786    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 03:55:18.046001 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.045985    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 03:55:18.046092 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.046022    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 03:55:18.046092 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.046049    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 03:55:18.046175 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.046090    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 03:55:18.048215 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.048197    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:18.116945 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.116893    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:18.117727 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.117712    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:18.117838 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.117745    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:18.117838 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.117761    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:18.117838 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.117797    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.127308 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.127293    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.127380 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.127312    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-244.ec2.internal\": node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.146271 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.146252    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"]
Jul 23 03:55:18.146363 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.146313    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:18.147061 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.147045    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:18.147139 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.147072    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:18.147139 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.147082    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:18.149585 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.149573    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:18.149703 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.149684    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.149703 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.149697    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.149794 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.149722    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:18.149794 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.149721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.150251 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150234    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:18.150326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150265    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:18.150326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150238    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:18.150326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150293    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:18.150326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150302    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:18.150326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.150276    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:18.152598 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.152585    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.152649 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.152609    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:18.153405 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.153387    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:18.153531 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.153419    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:18.153531 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.153431    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:18.161398 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.161384    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.184474 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.184452    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-244.ec2.internal\" not found" node="ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.188972 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.188956    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-244.ec2.internal\" not found" node="ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.250470 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.250445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.250470 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.250470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.250597 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.250491    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11d946897901f8dd71884d5e0994bb9a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"11d946897901f8dd71884d5e0994bb9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.250597 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.250538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.250597 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.250541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7dfc372bee77108c42583b8d28fdf77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"b7dfc372bee77108c42583b8d28fdf77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.261579 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.261554    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.351331 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.351291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11d946897901f8dd71884d5e0994bb9a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"11d946897901f8dd71884d5e0994bb9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.351441 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.351356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11d946897901f8dd71884d5e0994bb9a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"11d946897901f8dd71884d5e0994bb9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.362442 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.362418    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.463322 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.463254    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.487439 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.487417    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.491318 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.491299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:18.563735 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.563704    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.664323 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.664299    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.764924 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.764893    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.805316 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.805287    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:18.866064 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.866032    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.881963 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.881939    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 03:55:18.882088 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.882069    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 03:55:18.882123 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.882101    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 03:55:18.967179 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:18.967146    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:18.975309 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.975268    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 03:50:18 +0000 UTC" deadline="2028-01-10 08:44:28.027495697 +0000 UTC"
Jul 23 03:55:18.975309 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:18.975296    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12868h49m9.052202727s"
Jul 23 03:55:19.041429 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.041357    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 03:55:19.043140 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.043111    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 03:55:19.043240 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.043162    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 03:55:19.043240 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.043179    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Jul 23 03:55:19.067748 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:19.067722    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:19.141288 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:19.141255    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7dfc372bee77108c42583b8d28fdf77.slice/crio-be8a3fbc47ddfd63d4c178a3e90cd33d505af4e8ec3346dc14132071eb96c358 WatchSource:0}: Error finding container be8a3fbc47ddfd63d4c178a3e90cd33d505af4e8ec3346dc14132071eb96c358: Status 404 returned error can't find the container with id be8a3fbc47ddfd63d4c178a3e90cd33d505af4e8ec3346dc14132071eb96c358
Jul 23 03:55:19.146012 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.145992    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 03:55:19.152009 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:19.151986    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d946897901f8dd71884d5e0994bb9a.slice/crio-0d098e6ed4d4c294590c7c79c07d296f0135d857ca1104ec05dee30ffc4ce2fe WatchSource:0}: Error finding container 0d098e6ed4d4c294590c7c79c07d296f0135d857ca1104ec05dee30ffc4ce2fe: Status 404 returned error can't find the container with id 0d098e6ed4d4c294590c7c79c07d296f0135d857ca1104ec05dee30ffc4ce2fe
Jul 23 03:55:19.168837 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:19.168815    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Jul 23 03:55:19.182988 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.182967    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:19.249247 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.249224    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:19.285491 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.285470    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 03:55:19.286660 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.286646    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Jul 23 03:55:19.309817 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.309755    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 03:55:19.324551 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.324530    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:19.927255 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.926962    2582 apiserver.go:52] "Watching apiserver"
Jul 23 03:55:19.937671 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.937642    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 03:55:19.938300 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.938272    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-xvgdl","kube-system/global-pull-secret-syncer-tzgz6","openshift-cluster-node-tuning-operator/tuned-f45r6","openshift-multus/multus-additional-cni-plugins-qk5n8","openshift-multus/multus-tlr2v","openshift-multus/network-metrics-daemon-snqz6","openshift-network-operator/iptables-alerter-6b8vk","openshift-ovn-kubernetes/ovnkube-node-rjk9l","kube-system/konnectivity-agent-8g4mn","kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v","openshift-dns/node-resolver-dd5td","openshift-image-registry/node-ca-nbptw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"]
Jul 23 03:55:19.941154 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.941130    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.943787 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.943762    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.944773 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.944755    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zgj4r\""
Jul 23 03:55:19.945134 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.944958    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.945134 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.944973    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 03:55:19.945134 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.945022    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 03:55:19.945134 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.945030    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.946387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.946101    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 03:55:19.947328 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.946814    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.947821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948051    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948248    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948290    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948299    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j4vp2\""
Jul 23 03:55:19.948495 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.948414    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.949446 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.949426    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.949539 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.949522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.949597 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.949562    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t4w9j\""
Jul 23 03:55:19.949692 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.949670    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 03:55:19.949835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.949817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.952160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.952140    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 03:55:19.952279 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.952214    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:19.952535 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.952428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.952604 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.952594    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nvwm8\""
Jul 23 03:55:19.952870 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.952818    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 03:55:19.955663 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.955640    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:19.955762 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.955684    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.955828 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:19.955753    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:19.956496 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.956471    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ds9tf\""
Jul 23 03:55:19.959604 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.957667    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.961830 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.961795    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:19.962000 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:19.961948    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:19.963595 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963573    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-script-lib\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.963704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-systemd-units\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.963704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-kubelet\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.963704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963661    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-cnibin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.963704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-node-log\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963709    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-run\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-netns\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-etc-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-netd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-modprobe-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.963948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963928    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-lib-modules\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.963976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-daemon-config\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovn-node-metrics-cert\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9r76\" (UniqueName: \"kubernetes.io/projected/421e07a0-2128-4f23-b8c1-830d1925ab0b-kube-api-access-z9r76\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-var-lib-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964115    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964154    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-bin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt6ks\" (UniqueName: \"kubernetes.io/projected/c0d5ab6e-455c-4e96-a0c0-27e43a398778-kube-api-access-pt6ks\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:19.964260 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-sys\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-etc-kubernetes\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d5ab6e-455c-4e96-a0c0-27e43a398778-host-slash\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964349    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964376    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-kubernetes\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-conf-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-system-cni-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-host\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-log-socket\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964651    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-os-release\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-cni-binary-copy\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-multus\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkdlx\" (UniqueName: \"kubernetes.io/projected/f2de0078-cf9c-4151-916a-92dbe3715ccc-kube-api-access-wkdlx\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-etc-tuned\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964744    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-ovn\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.964759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-bin\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qf58\" (UniqueName: \"kubernetes.io/projected/e12ac667-351b-4b60-b8f3-ad975c267442-kube-api-access-6qf58\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-env-overrides\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cnibin\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-binary-copy\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-systemd\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-socket-dir-parent\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964922    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-k8s-cni-cncf-io\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-netns\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.964983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-hostroot\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965054    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965080    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0d5ab6e-455c-4e96-a0c0-27e43a398778-iptables-alerter-script\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysconfig\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-multus-certs\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-slash\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.965623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-os-release\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-var-lib-kubelet\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-system-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965312    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-kubelet\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965336    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-systemd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965425    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-conf\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-config\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qct9k\" (UniqueName: \"kubernetes.io/projected/65fd0ded-4df0-481e-8e96-cbddc1fdb592-kube-api-access-qct9k\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-tmp\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.965683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:19.966417 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:19.965750    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:19.968273 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.967738    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:19.970612 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.970591    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 03:55:19.971274 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.971004    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:19.971274 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.971084    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 03:55:19.971274 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.971093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5vkwf\""
Jul 23 03:55:19.971763 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.971601    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 03:55:19.973634 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.973400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:19.973982 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.973958    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 03:55:19.974142 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.974094    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gz7wj\""
Jul 23 03:55:19.974217 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.974176    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.974274 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.974220    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.975983 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.975963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:19.976675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.976653    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.976761 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.976725    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hwrkm\""
Jul 23 03:55:19.976977 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.976958    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.978486 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.978467    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q7pb9\""
Jul 23 03:55:19.978609 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.978583    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 03:55:19.978785 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.978766    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 03:55:19.978785 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.978781    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 03:55:19.996095 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:19.996074    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fsdws"
Jul 23 03:55:20.002944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.002920    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fsdws"
Jul 23 03:55:20.052716 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.052302    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 03:55:20.053056 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.052994    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" event={"ID":"11d946897901f8dd71884d5e0994bb9a","Type":"ContainerStarted","Data":"0d098e6ed4d4c294590c7c79c07d296f0135d857ca1104ec05dee30ffc4ce2fe"}
Jul 23 03:55:20.054849 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.054816    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"b7dfc372bee77108c42583b8d28fdf77","Type":"ContainerStarted","Data":"be8a3fbc47ddfd63d4c178a3e90cd33d505af4e8ec3346dc14132071eb96c358"}
Jul 23 03:55:20.065766 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-netns\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.065909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-hostroot\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.065909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.065909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0d5ab6e-455c-4e96-a0c0-27e43a398778-iptables-alerter-script\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.065909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.065909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysconfig\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065944    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-multus-certs\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-slash\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.065992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-os-release\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-var-lib-kubelet\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-system-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-kubelet\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-systemd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-conf\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066158    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:20.066200 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-kubelet-config\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-config\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qct9k\" (UniqueName: \"kubernetes.io/projected/65fd0ded-4df0-481e-8e96-cbddc1fdb592-kube-api-access-qct9k\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-slash\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-tmp\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-device-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-script-lib\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-systemd-units\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-kubelet\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-cnibin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-node-log\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-run\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-konnectivity-ca\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-netns\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-etc-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066588    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-netd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.066751 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066636    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-modprobe-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-lib-modules\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqlt\" (UniqueName: \"kubernetes.io/projected/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-kube-api-access-8lqlt\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-daemon-config\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovn-node-metrics-cert\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9r76\" (UniqueName: \"kubernetes.io/projected/421e07a0-2128-4f23-b8c1-830d1925ab0b-kube-api-access-z9r76\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xfb\" (UniqueName: \"kubernetes.io/projected/78a35807-3b44-4fab-b9ea-e877402a4551-kube-api-access-t2xfb\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-var-lib-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-bin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.066980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067009    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt6ks\" (UniqueName: \"kubernetes.io/projected/c0d5ab6e-455c-4e96-a0c0-27e43a398778-kube-api-access-pt6ks\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-sys\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8tn9\" (UniqueName: \"kubernetes.io/projected/71ede43b-4d5c-4ee5-87c0-384f67a7509d-kube-api-access-h8tn9\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-registration-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.067675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-netns\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-etc-kubernetes\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-etc-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067194    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d5ab6e-455c-4e96-a0c0-27e43a398778-host-slash\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-conf\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-netd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-kubernetes\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067756    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-run\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067814    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-systemd-units\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-kubelet\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-cnibin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067952    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-script-lib\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067983    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-node-log\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-etc-kubernetes\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068035    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d5ab6e-455c-4e96-a0c0-27e43a398778-host-slash\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.067954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-daemon-config\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.068387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068062    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-modprobe-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068113    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-system-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-os-release\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068156    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysctl-d\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068166    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-var-lib-kubelet\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068213    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-systemd\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068254    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-bin\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-sys-fs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068276    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-sys\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dcgs\" (UniqueName: \"kubernetes.io/projected/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kube-api-access-8dcgs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068323    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-agent-certs\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-conf-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068434    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-system-cni-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.069017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068491    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-host\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068536    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-socket-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovnkube-config\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-sysconfig\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-kubernetes\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-log-socket\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-os-release\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-cni-binary-copy\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-multus\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068791    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-lib-modules\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkdlx\" (UniqueName: \"kubernetes.io/projected/f2de0078-cf9c-4151-916a-92dbe3715ccc-kube-api-access-wkdlx\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068809    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0d5ab6e-455c-4e96-a0c0-27e43a398778-iptables-alerter-script\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-etc-tuned\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068865    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-cni-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068931    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-var-lib-openvswitch\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71ede43b-4d5c-4ee5-87c0-384f67a7509d-hosts-file\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.069927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068970    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-log-socket\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068966    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-ovn\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.068971    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-conf-dir\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069028    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-var-lib-cni-multus\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-bin\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069065    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-kubelet\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-run-netns\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qf58\" (UniqueName: \"kubernetes.io/projected/e12ac667-351b-4b60-b8f3-ad975c267442-kube-api-access-6qf58\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-serviceca\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069138    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-os-release\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-dbus\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069177    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-env-overrides\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-multus-certs\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cnibin\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.072756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-host-cni-bin\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-binary-copy\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-systemd\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-host\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-socket-dir-parent\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-k8s-cni-cncf-io\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71ede43b-4d5c-4ee5-87c0-384f67a7509d-tmp-dir\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069525    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2de0078-cf9c-4151-916a-92dbe3715ccc-cni-binary-copy\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cnibin\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-hostroot\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069751    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65fd0ded-4df0-481e-8e96-cbddc1fdb592-cni-binary-copy\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-etc-systemd\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069812    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65fd0ded-4df0-481e-8e96-cbddc1fdb592-system-cni-dir\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-multus-socket-dir-parent\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2de0078-cf9c-4151-916a-92dbe3715ccc-host-run-k8s-cni-cncf-io\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/421e07a0-2128-4f23-b8c1-830d1925ab0b-run-ovn\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069934    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e12ac667-351b-4b60-b8f3-ad975c267442-host\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.069964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/421e07a0-2128-4f23-b8c1-830d1925ab0b-env-overrides\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.073690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.070795    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-tmp\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.074413 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.071707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/421e07a0-2128-4f23-b8c1-830d1925ab0b-ovn-node-metrics-cert\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.074413 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.072275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e12ac667-351b-4b60-b8f3-ad975c267442-etc-tuned\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.082484 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.082458    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qct9k\" (UniqueName: \"kubernetes.io/projected/65fd0ded-4df0-481e-8e96-cbddc1fdb592-kube-api-access-qct9k\") pod \"multus-additional-cni-plugins-qk5n8\" (UID: \"65fd0ded-4df0-481e-8e96-cbddc1fdb592\") " pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.083207 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.083118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt6ks\" (UniqueName: \"kubernetes.io/projected/c0d5ab6e-455c-4e96-a0c0-27e43a398778-kube-api-access-pt6ks\") pod \"iptables-alerter-6b8vk\" (UID: \"c0d5ab6e-455c-4e96-a0c0-27e43a398778\") " pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.083207 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.083172    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qf58\" (UniqueName: \"kubernetes.io/projected/e12ac667-351b-4b60-b8f3-ad975c267442-kube-api-access-6qf58\") pod \"tuned-f45r6\" (UID: \"e12ac667-351b-4b60-b8f3-ad975c267442\") " pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.083591 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.083562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkdlx\" (UniqueName: \"kubernetes.io/projected/f2de0078-cf9c-4151-916a-92dbe3715ccc-kube-api-access-wkdlx\") pod \"multus-tlr2v\" (UID: \"f2de0078-cf9c-4151-916a-92dbe3715ccc\") " pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.084267 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.084243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9r76\" (UniqueName: \"kubernetes.io/projected/421e07a0-2128-4f23-b8c1-830d1925ab0b-kube-api-access-z9r76\") pod \"ovnkube-node-rjk9l\" (UID: \"421e07a0-2128-4f23-b8c1-830d1925ab0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.170498 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170466    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqlt\" (UniqueName: \"kubernetes.io/projected/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-kube-api-access-8lqlt\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170510    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xfb\" (UniqueName: \"kubernetes.io/projected/78a35807-3b44-4fab-b9ea-e877402a4551-kube-api-access-t2xfb\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8tn9\" (UniqueName: \"kubernetes.io/projected/71ede43b-4d5c-4ee5-87c0-384f67a7509d-kube-api-access-h8tn9\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170550    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-registration-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-sys-fs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dcgs\" (UniqueName: \"kubernetes.io/projected/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kube-api-access-8dcgs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-agent-certs\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-registration-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.170679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-socket-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71ede43b-4d5c-4ee5-87c0-384f67a7509d-hosts-file\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-serviceca\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170845    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170854    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-sys-fs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-dbus\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.170944    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170950    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71ede43b-4d5c-4ee5-87c0-384f67a7509d-hosts-file\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.170970    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-socket-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-host\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-dbus\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.171040    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:20.670999392 +0000 UTC m=+3.132143058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-host\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.171107 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71ede43b-4d5c-4ee5-87c0-384f67a7509d-tmp-dir\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-kubelet-config\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-device-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-konnectivity-ca\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58af239e-1d0b-472a-8dc7-ddc26061a37a-kubelet-config\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.171369    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.171416    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:20.67140077 +0000 UTC m=+3.132544423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-serviceca\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171438    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-device-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171438    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71ede43b-4d5c-4ee5-87c0-384f67a7509d-tmp-dir\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.171835 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.171474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.172582 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.172562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-konnectivity-ca\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.173205 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.173185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6-agent-certs\") pod \"konnectivity-agent-8g4mn\" (UID: \"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6\") " pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.177860 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.177800    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:20.177860 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.177830    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:20.177860 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.177849    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:20.178073 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.177942    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:20.677922404 +0000 UTC m=+3.139066059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:20.180500 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.180476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xfb\" (UniqueName: \"kubernetes.io/projected/78a35807-3b44-4fab-b9ea-e877402a4551-kube-api-access-t2xfb\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.180813 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.180794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dcgs\" (UniqueName: \"kubernetes.io/projected/adb327dc-5a8a-4343-8e23-1bd92cf96c61-kube-api-access-8dcgs\") pod \"aws-ebs-csi-driver-node-nx28v\" (UID: \"adb327dc-5a8a-4343-8e23-1bd92cf96c61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.180912 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.180812    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8tn9\" (UniqueName: \"kubernetes.io/projected/71ede43b-4d5c-4ee5-87c0-384f67a7509d-kube-api-access-h8tn9\") pod \"node-resolver-dd5td\" (UID: \"71ede43b-4d5c-4ee5-87c0-384f67a7509d\") " pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.180912 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.180816    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqlt\" (UniqueName: \"kubernetes.io/projected/09f1bdd1-e10c-4253-8f7c-1606d3e0b58a-kube-api-access-8lqlt\") pod \"node-ca-nbptw\" (UID: \"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a\") " pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.257583 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.257546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tlr2v"
Jul 23 03:55:20.269257 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.269236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:20.287713 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.287690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6b8vk"
Jul 23 03:55:20.293639 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.293617    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qk5n8"
Jul 23 03:55:20.301578 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.301559    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f45r6"
Jul 23 03:55:20.309460 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.309444    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:20.317282 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.317245    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v"
Jul 23 03:55:20.325057 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.325036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dd5td"
Jul 23 03:55:20.331951 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.331935    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nbptw"
Jul 23 03:55:20.675725 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.675674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:20.675963 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.675748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:20.675963 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.675870    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:20.675963 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.675913    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:20.675963 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.675954    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:21.675934644 +0000 UTC m=+4.137078301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:20.676173 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.675971    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:21.675964095 +0000 UTC m=+4.137107747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:20.776330 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:20.776291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:20.776518 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.776477    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:20.776518 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.776505    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:20.776632 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.776521    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:20.776632 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:20.776577    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:21.776561409 +0000 UTC m=+4.237705061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:21.006920 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.006875    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 03:50:20 +0000 UTC" deadline="2028-03-10 04:10:41.186900764 +0000 UTC"
Jul 23 03:55:21.006920 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.006917    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14304h15m20.179989794s"
Jul 23 03:55:21.049662 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.049627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:21.049858 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.049783    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:21.218976 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.218950    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfaa5c1e_ed91_43d8_92c3_ceef77b14ec6.slice/crio-b5bbcefd08f04dc86d79e32e6b8e7fb633110f51bcf828d09ec867127ee68610 WatchSource:0}: Error finding container b5bbcefd08f04dc86d79e32e6b8e7fb633110f51bcf828d09ec867127ee68610: Status 404 returned error can't find the container with id b5bbcefd08f04dc86d79e32e6b8e7fb633110f51bcf828d09ec867127ee68610
Jul 23 03:55:21.221021 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.220991    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2de0078_cf9c_4151_916a_92dbe3715ccc.slice/crio-ae96d5c42bb7c8a5615b02184f6992df07cfa586f50c12f48c8ce1a01fef3f6d WatchSource:0}: Error finding container ae96d5c42bb7c8a5615b02184f6992df07cfa586f50c12f48c8ce1a01fef3f6d: Status 404 returned error can't find the container with id ae96d5c42bb7c8a5615b02184f6992df07cfa586f50c12f48c8ce1a01fef3f6d
Jul 23 03:55:21.221762 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.221659    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65fd0ded_4df0_481e_8e96_cbddc1fdb592.slice/crio-a7a5192733fc8ed9ff7215115264a0d8168eb57b9156fe9b54090f57e8467b01 WatchSource:0}: Error finding container a7a5192733fc8ed9ff7215115264a0d8168eb57b9156fe9b54090f57e8467b01: Status 404 returned error can't find the container with id a7a5192733fc8ed9ff7215115264a0d8168eb57b9156fe9b54090f57e8467b01
Jul 23 03:55:21.222962 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.222880    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d5ab6e_455c_4e96_a0c0_27e43a398778.slice/crio-1fdd2e9cd99554a58734f9fe7224556b412f9d97f94dfd4e962d93b171bc1401 WatchSource:0}: Error finding container 1fdd2e9cd99554a58734f9fe7224556b412f9d97f94dfd4e962d93b171bc1401: Status 404 returned error can't find the container with id 1fdd2e9cd99554a58734f9fe7224556b412f9d97f94dfd4e962d93b171bc1401
Jul 23 03:55:21.223642 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.223611    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12ac667_351b_4b60_b8f3_ad975c267442.slice/crio-e18988e9e08ec40feb5f082ba3458ce8eaba36e51eb9773d90d0b0b729899dad WatchSource:0}: Error finding container e18988e9e08ec40feb5f082ba3458ce8eaba36e51eb9773d90d0b0b729899dad: Status 404 returned error can't find the container with id e18988e9e08ec40feb5f082ba3458ce8eaba36e51eb9773d90d0b0b729899dad
Jul 23 03:55:21.249325 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.249294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421e07a0_2128_4f23_b8c1_830d1925ab0b.slice/crio-f78de348e28e858653ffddc2685c5e7d482ca784441b7fa2c3c9ef04f2cf2828 WatchSource:0}: Error finding container f78de348e28e858653ffddc2685c5e7d482ca784441b7fa2c3c9ef04f2cf2828: Status 404 returned error can't find the container with id f78de348e28e858653ffddc2685c5e7d482ca784441b7fa2c3c9ef04f2cf2828
Jul 23 03:55:21.251110 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.251086    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f1bdd1_e10c_4253_8f7c_1606d3e0b58a.slice/crio-54fe18f1510f3f0b05df33e9b6a440c57fae86d607c10bdd8364212f7f9f0704 WatchSource:0}: Error finding container 54fe18f1510f3f0b05df33e9b6a440c57fae86d607c10bdd8364212f7f9f0704: Status 404 returned error can't find the container with id 54fe18f1510f3f0b05df33e9b6a440c57fae86d607c10bdd8364212f7f9f0704
Jul 23 03:55:21.252070 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.251978    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb327dc_5a8a_4343_8e23_1bd92cf96c61.slice/crio-56a3cb6b9eee8203b57a65dfb323c3904b2f293ff7c05ff1e96b47b992d9189c WatchSource:0}: Error finding container 56a3cb6b9eee8203b57a65dfb323c3904b2f293ff7c05ff1e96b47b992d9189c: Status 404 returned error can't find the container with id 56a3cb6b9eee8203b57a65dfb323c3904b2f293ff7c05ff1e96b47b992d9189c
Jul 23 03:55:21.254224 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:21.254202    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ede43b_4d5c_4ee5_87c0_384f67a7509d.slice/crio-c6c9939f0bc97425123b3e3f4408fa1f1ae1d06e95fd7d3ceb4a1a38bc452fda WatchSource:0}: Error finding container c6c9939f0bc97425123b3e3f4408fa1f1ae1d06e95fd7d3ceb4a1a38bc452fda: Status 404 returned error can't find the container with id c6c9939f0bc97425123b3e3f4408fa1f1ae1d06e95fd7d3ceb4a1a38bc452fda
Jul 23 03:55:21.307515 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.307310    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 03:55:21.683686 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.683633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:21.683897 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.683724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:21.683897 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.683834    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:21.683897 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.683861    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:21.684092 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.683914    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:23.683888456 +0000 UTC m=+6.145032134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:21.684092 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.683935    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:23.683924946 +0000 UTC m=+6.145068599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:21.785024 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:21.784942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:21.785199 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.785123    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:21.785199 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.785146    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:21.785199 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.785159    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:21.785359 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:21.785220    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:23.785200668 +0000 UTC m=+6.246344344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:22.007509 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.007436    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 03:50:20 +0000 UTC" deadline="2028-04-18 00:56:58.380825038 +0000 UTC"
Jul 23 03:55:22.007509 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.007508    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15237h1m36.373322298s"
Jul 23 03:55:22.047956 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.046998    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:22.047956 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:22.047154    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:22.047956 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.047271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:22.047956 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:22.047496    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:22.064309 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.064273    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8g4mn" event={"ID":"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6","Type":"ContainerStarted","Data":"b5bbcefd08f04dc86d79e32e6b8e7fb633110f51bcf828d09ec867127ee68610"}
Jul 23 03:55:22.080927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.080365    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" event={"ID":"11d946897901f8dd71884d5e0994bb9a","Type":"ContainerStarted","Data":"70f4c81fa498ad5b4a192edf7bb1a21ed2433353d038d80faf6bdd8ede501c03"}
Jul 23 03:55:22.086798 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.086081    2582 generic.go:356] "Generic (PLEG): container finished" podID="b7dfc372bee77108c42583b8d28fdf77" containerID="138f8811b68824ebddb7c6ed3809613d1a867323329530b3946624a86da4c08e" exitCode=0
Jul 23 03:55:22.086798 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.086169    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"b7dfc372bee77108c42583b8d28fdf77","Type":"ContainerDied","Data":"138f8811b68824ebddb7c6ed3809613d1a867323329530b3946624a86da4c08e"}
Jul 23 03:55:22.101294 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.101227    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dd5td" event={"ID":"71ede43b-4d5c-4ee5-87c0-384f67a7509d","Type":"ContainerStarted","Data":"c6c9939f0bc97425123b3e3f4408fa1f1ae1d06e95fd7d3ceb4a1a38bc452fda"}
Jul 23 03:55:22.113605 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.113539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v" event={"ID":"adb327dc-5a8a-4343-8e23-1bd92cf96c61","Type":"ContainerStarted","Data":"56a3cb6b9eee8203b57a65dfb323c3904b2f293ff7c05ff1e96b47b992d9189c"}
Jul 23 03:55:22.116439 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.116005    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" podStartSLOduration=2.115986025 podStartE2EDuration="2.115986025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:22.097427325 +0000 UTC m=+4.558571002" watchObservedRunningTime="2026-07-23 03:55:22.115986025 +0000 UTC m=+4.577129719"
Jul 23 03:55:22.130552 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.130232    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"f78de348e28e858653ffddc2685c5e7d482ca784441b7fa2c3c9ef04f2cf2828"}
Jul 23 03:55:22.134361 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.134281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6b8vk" event={"ID":"c0d5ab6e-455c-4e96-a0c0-27e43a398778","Type":"ContainerStarted","Data":"1fdd2e9cd99554a58734f9fe7224556b412f9d97f94dfd4e962d93b171bc1401"}
Jul 23 03:55:22.164039 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.163948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerStarted","Data":"a7a5192733fc8ed9ff7215115264a0d8168eb57b9156fe9b54090f57e8467b01"}
Jul 23 03:55:22.182830 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.182555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tlr2v" event={"ID":"f2de0078-cf9c-4151-916a-92dbe3715ccc","Type":"ContainerStarted","Data":"ae96d5c42bb7c8a5615b02184f6992df07cfa586f50c12f48c8ce1a01fef3f6d"}
Jul 23 03:55:22.195752 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.195674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nbptw" event={"ID":"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a","Type":"ContainerStarted","Data":"54fe18f1510f3f0b05df33e9b6a440c57fae86d607c10bdd8364212f7f9f0704"}
Jul 23 03:55:22.215691 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:22.215631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f45r6" event={"ID":"e12ac667-351b-4b60-b8f3-ad975c267442","Type":"ContainerStarted","Data":"e18988e9e08ec40feb5f082ba3458ce8eaba36e51eb9773d90d0b0b729899dad"}
Jul 23 03:55:23.047358 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:23.047322    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:23.047932 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.047460    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:23.234988 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:23.234876    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"b7dfc372bee77108c42583b8d28fdf77","Type":"ContainerStarted","Data":"db1d01e468a40b2a2f5f10296a8ed1c3157400b625597545a79d39fe574e3866"}
Jul 23 03:55:23.702124 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:23.702080    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:23.702326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:23.702182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:23.702401 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.702322    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:23.702401 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.702393    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:27.702370996 +0000 UTC m=+10.163514653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:23.702852 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.702829    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:23.702975 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.702908    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:27.702874104 +0000 UTC m=+10.164017757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:23.803346 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:23.803293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:23.803539 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.803500    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:23.803539 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.803525    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:23.803539 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.803539    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:23.803718 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:23.803613    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:27.803592423 +0000 UTC m=+10.264736081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:24.049035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:24.049005    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:24.049518 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:24.049143    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:24.049586 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:24.049557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:24.049691 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:24.049669    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:25.047035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:25.046992    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:25.047216 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:25.047126    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:26.047679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:26.046938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:26.047679 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:26.047091    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:26.047679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:26.047532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:26.047679 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:26.047637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:27.047218 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:27.047183    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:27.047399 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.047324    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:27.737263 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:27.737225    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:27.737759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:27.737292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:27.737759 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.737441    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:27.737759 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.737519    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:35.737498797 +0000 UTC m=+18.198642465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:27.737759 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.737441    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:27.737759 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.737574    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:35.737562359 +0000 UTC m=+18.198706011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:27.838236 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:27.838197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:27.838436 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.838409    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:27.838527 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.838445    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:27.838527 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.838465    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:27.838671 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:27.838527    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:35.838509103 +0000 UTC m=+18.299652777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:28.047500 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:28.047419    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:28.047674 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:28.047513    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:28.047674 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:28.047614    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:28.047802 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:28.047662    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:29.047288 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:29.047251    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:29.047729 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:29.047391    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:30.047904 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:30.047587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:30.047904 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:30.047717    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:30.048400 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:30.048362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:30.049665 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:30.048498    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:31.047250 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:31.047219    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:31.047419 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:31.047341    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:32.046678 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:32.046644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:32.047132 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:32.046692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:32.047132 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:32.046787    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:32.047132 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:32.046887    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:33.046770 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:33.046740    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:33.047165 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:33.046852    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:34.046874 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:34.046844    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:34.047297 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:34.046882    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:34.047297 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:34.046985    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:34.047297 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:34.047138    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:35.046485 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:35.046453    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:35.046696 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.046560    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:35.798867 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:35.798829    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:35.799342 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:35.798887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:35.799342 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.798976    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:35.799342 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.798993    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:35.799342 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.799052    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:55:51.799030867 +0000 UTC m=+34.260174533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:35.799342 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.799074    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:51.799061429 +0000 UTC m=+34.260205083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:35.899398 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:35.899369    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:35.899527 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.899508    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:35.899601 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.899527    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:35.899601 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.899538    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:35.899601 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:35.899583    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:51.899570244 +0000 UTC m=+34.360713894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:36.046755 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:36.046727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:36.046944 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:36.046870    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:36.047004 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:36.046965    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:36.047109 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:36.047087    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:37.046577 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:37.046546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:37.047037 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:37.046656    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:38.048039 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:38.048003    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:38.048445 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:38.048113    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:38.048445 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:38.048148    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:38.048445 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:38.048233    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:39.046306 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:39.046280    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:39.046467 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:39.046392    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:40.046630 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:40.046605    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:40.047039 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:40.046718    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:40.047039 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:40.046792    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:40.047039 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:40.046891    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:41.047183 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.046900    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:41.047699 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:41.047255    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:41.277543 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.277514    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tlr2v" event={"ID":"f2de0078-cf9c-4151-916a-92dbe3715ccc","Type":"ContainerStarted","Data":"5d0e941536935c29f6ef65addbdac6c5fe8c8f3c777ba8110ae5c481de6c2136"}
Jul 23 03:55:41.279090 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.279063    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nbptw" event={"ID":"09f1bdd1-e10c-4253-8f7c-1606d3e0b58a","Type":"ContainerStarted","Data":"9634a7d9513dcd6e42cfe9d7ead97d64edaffe464576f1fc527bd66dc1a46988"}
Jul 23 03:55:41.280517 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.280484    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f45r6" event={"ID":"e12ac667-351b-4b60-b8f3-ad975c267442","Type":"ContainerStarted","Data":"66be26fb28aa242394d130fa65ee15eae2758fa432a7440c02218981cc206c1a"}
Jul 23 03:55:41.282031 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.282010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8g4mn" event={"ID":"dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6","Type":"ContainerStarted","Data":"5dc658755503d474ef3a6061923751daf65f7f748b36fca7b36a2a3fd734bb0f"}
Jul 23 03:55:41.283511 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.283492    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dd5td" event={"ID":"71ede43b-4d5c-4ee5-87c0-384f67a7509d","Type":"ContainerStarted","Data":"9783d164716d54797e66d79f0800f2876f43da2189e9bf0c8c6908cfdbdb610c"}
Jul 23 03:55:41.284970 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.284949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v" event={"ID":"adb327dc-5a8a-4343-8e23-1bd92cf96c61","Type":"ContainerStarted","Data":"e51685e0c56ad3e460134ac662bd0dac0a7a3b43a9d6907e33481a5b9d5281b4"}
Jul 23 03:55:41.287634 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"8a85ee47a101351a759fd41079058065794ced34d1a673ce93b89336920edf28"}
Jul 23 03:55:41.287717 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"4fcd8c4556d6f660b5514462a541eef6d142f5ece74359ce679e71cbe43572f1"}
Jul 23 03:55:41.287717 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"f10c327d09e15214d4516104385770908d0366ba36da58a8c670823433124d76"}
Jul 23 03:55:41.287827 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287749    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"4cb38c4a75b17132c440b63edb7685f8cfcf1c4f200d66bab7b7d2384f5e5038"}
Jul 23 03:55:41.287827 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287781    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"2e05fa7b72c8f69d2f7e2bdd11e38473b2a0088553457e483d405d82826f11b0"}
Jul 23 03:55:41.287913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.287832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"f14a9035aea177f03a5b7a8d61ced938ac42fdb4294e17d659c4460f402c70ec"}
Jul 23 03:55:41.288999 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.288978    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="f6da22074a7fe8154f06f75d590e67df0f2994981428c12aecd181ee90fdf011" exitCode=0
Jul 23 03:55:41.289096 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.289006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"f6da22074a7fe8154f06f75d590e67df0f2994981428c12aecd181ee90fdf011"}
Jul 23 03:55:41.294024 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.293985    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" podStartSLOduration=21.29397231 podStartE2EDuration="21.29397231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:23.252804483 +0000 UTC m=+5.713948159" watchObservedRunningTime="2026-07-23 03:55:41.29397231 +0000 UTC m=+23.755115982"
Jul 23 03:55:41.294278 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.294252    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tlr2v" podStartSLOduration=4.009199647 podStartE2EDuration="23.294244635s" totalImagesPullingTime="19.285044988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:41.293704439 +0000 UTC m=+23.754848110" watchObservedRunningTime="2026-07-23 03:55:41.294244635 +0000 UTC m=+23.755388307"
Jul 23 03:55:41.337077 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.337035    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f45r6" podStartSLOduration=3.138789564 podStartE2EDuration="22.337021946s" totalImagesPullingTime="19.198232382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:41.33688822 +0000 UTC m=+23.798031912" watchObservedRunningTime="2026-07-23 03:55:41.337021946 +0000 UTC m=+23.798165621"
Jul 23 03:55:41.383017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.382972    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dd5td" podStartSLOduration=3.194093931 podStartE2EDuration="22.382953447s" totalImagesPullingTime="19.188859516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:41.382502 +0000 UTC m=+23.843645671" watchObservedRunningTime="2026-07-23 03:55:41.382953447 +0000 UTC m=+23.844097124"
Jul 23 03:55:41.383784 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.383754    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8g4mn" podStartSLOduration=11.278104159 podStartE2EDuration="22.383741028s" totalImagesPullingTime="11.105636869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:41.358158372 +0000 UTC m=+23.819302045" watchObservedRunningTime="2026-07-23 03:55:41.383741028 +0000 UTC m=+23.844884701"
Jul 23 03:55:41.402738 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.402704    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nbptw" podStartSLOduration=3.2109315880000002 podStartE2EDuration="22.402688546s" totalImagesPullingTime="19.191756958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:41.402183262 +0000 UTC m=+23.863326937" watchObservedRunningTime="2026-07-23 03:55:41.402688546 +0000 UTC m=+23.863832236"
Jul 23 03:55:41.902887 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:41.902750    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 03:55:42.031447 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.031349    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T03:55:41.902766673Z","UUID":"3e1c8405-bd8d-40ec-a0eb-e98260a346aa","Handler":null,"Name":"","Endpoint":""}
Jul 23 03:55:42.032770 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.032749    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 03:55:42.032911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.032795    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 03:55:42.046672 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.046644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:42.046782 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.046649    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:42.046854 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:42.046776    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:42.046909 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:42.046851    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:42.291932 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.291901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v" event={"ID":"adb327dc-5a8a-4343-8e23-1bd92cf96c61","Type":"ContainerStarted","Data":"b18569e0e1623af1860088bd2bbb4d58b53d573f0f8d044c3abbda6971c73679"}
Jul 23 03:55:42.293138 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.293110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6b8vk" event={"ID":"c0d5ab6e-455c-4e96-a0c0-27e43a398778","Type":"ContainerStarted","Data":"d7cef7fa939572f1e028e5cdc68e65244abc82e354bfece96ef268de06bd9125"}
Jul 23 03:55:42.308480 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.308444    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6b8vk" podStartSLOduration=4.110802526 podStartE2EDuration="23.308429166s" totalImagesPullingTime="19.19762664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:42.307902306 +0000 UTC m=+24.769045978" watchObservedRunningTime="2026-07-23 03:55:42.308429166 +0000 UTC m=+24.769572837"
Jul 23 03:55:42.490355 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.490324    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:42.490929 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:42.490911    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:43.046942 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.046911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:43.047144 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:43.047033    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:43.297531 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.297308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v" event={"ID":"adb327dc-5a8a-4343-8e23-1bd92cf96c61","Type":"ContainerStarted","Data":"a761bcf0d99a6613aebe9df294b6f00298ba1d14ad3376b75ad122ef840018f7"}
Jul 23 03:55:43.300670 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.300643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"158f7786d67504368c9920f6cb89591f8751e2d4b7a006f783b0cf548f8d2d6c"}
Jul 23 03:55:43.300857 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.300832    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:43.301344 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.301327    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8g4mn"
Jul 23 03:55:43.324326 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:43.324272    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx28v" podStartSLOduration=2.7649940170000002 podStartE2EDuration="24.32425584s" totalImagesPullingTime="21.559261823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:43.323433467 +0000 UTC m=+25.784577166" watchObservedRunningTime="2026-07-23 03:55:43.32425584 +0000 UTC m=+25.785399512"
Jul 23 03:55:44.047212 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:44.047162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:44.047422 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:44.047171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:44.047422 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:44.047285    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:44.047422 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:44.047381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:45.046822 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:45.046781    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:45.047335 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:45.046920    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:46.047248 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.047056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:46.047831 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.047112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:46.047831 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:46.047364    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:46.047831 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:46.047460    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:46.310981 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.310948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" event={"ID":"421e07a0-2128-4f23-b8c1-830d1925ab0b","Type":"ContainerStarted","Data":"d98d7fbcf915989132eb15f3aa409a190a5e07e32823a486ae75b6ecde020d79"}
Jul 23 03:55:46.311302 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.311256    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:46.311397 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.311329    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:46.311397 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.311376    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:46.313075 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.313054    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="68edb007e56240420d69a7851aa4abf061aabe24cd4ebaac8c753dd62503ed5d" exitCode=0
Jul 23 03:55:46.313164 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.313095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"68edb007e56240420d69a7851aa4abf061aabe24cd4ebaac8c753dd62503ed5d"}
Jul 23 03:55:46.326341 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.326316    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:46.326584 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.326561    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:55:46.339612 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:46.339578    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l" podStartSLOduration=9.085425698 podStartE2EDuration="28.339566369s" totalImagesPullingTime="19.254140671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:46.339079745 +0000 UTC m=+28.800223431" watchObservedRunningTime="2026-07-23 03:55:46.339566369 +0000 UTC m=+28.800710042"
Jul 23 03:55:47.046781 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.046741    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:47.046981 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:47.046891    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:47.799286 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.799254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tzgz6"]
Jul 23 03:55:47.799645 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.799355    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:47.799645 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:47.799430    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:47.802062 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.802036    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xvgdl"]
Jul 23 03:55:47.802185 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.802126    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:47.802247 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:47.802223    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:47.802623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.802588    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-snqz6"]
Jul 23 03:55:47.802729 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:47.802680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:47.802779 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:47.802758    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:48.318126 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:48.318094    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="c7f38ceb077edb70cf1fbc9fd47564117f4530ad7d7bd09778ff43a5962c744c" exitCode=0
Jul 23 03:55:48.318255 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:48.318173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"c7f38ceb077edb70cf1fbc9fd47564117f4530ad7d7bd09778ff43a5962c744c"}
Jul 23 03:55:49.047008 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:49.046866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:49.047349 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:49.047090    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:49.322674 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:49.322648    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="b15938c79ec38039fd162f9c2814b2cfefb9dcc8c99e2b75666044ede61d237e" exitCode=0
Jul 23 03:55:49.322775 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:49.322709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"b15938c79ec38039fd162f9c2814b2cfefb9dcc8c99e2b75666044ede61d237e"}
Jul 23 03:55:50.047000 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:50.046964    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:50.047153 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:50.047101    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:50.047702 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:50.047153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:50.047702 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:50.047270    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:51.046628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:51.046593    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:51.046827 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.046715    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:51.813249 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:51.813218    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:51.813705 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:51.813279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:51.813705 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.813363    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:51.813705 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.813418    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:51.813705 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.813441    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret podName:58af239e-1d0b-472a-8dc7-ddc26061a37a nodeName:}" failed. No retries permitted until 2026-07-23 03:56:23.813426193 +0000 UTC m=+66.274569847 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret") pod "global-pull-secret-syncer-tzgz6" (UID: "58af239e-1d0b-472a-8dc7-ddc26061a37a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 03:55:51.813705 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.813471    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs podName:78a35807-3b44-4fab-b9ea-e877402a4551 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:23.813455103 +0000 UTC m=+66.274598757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs") pod "network-metrics-daemon-snqz6" (UID: "78a35807-3b44-4fab-b9ea-e877402a4551") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 03:55:51.914083 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:51.914052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:51.914229 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.914215    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 03:55:51.914272 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.914238    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 03:55:51.914272 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.914249    2582 projected.go:196] Error preparing data for projected volume kube-api-access-lvnt8 for pod openshift-network-diagnostics/network-check-target-xvgdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:51.914349 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:51.914303    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8 podName:0d412971-b53a-4379-b9ee-3f347dca0b61 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:23.914283938 +0000 UTC m=+66.375427588 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvnt8" (UniqueName: "kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8") pod "network-check-target-xvgdl" (UID: "0d412971-b53a-4379-b9ee-3f347dca0b61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 03:55:52.050235 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:52.050155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:52.050386 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:52.050258    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tzgz6" podUID="58af239e-1d0b-472a-8dc7-ddc26061a37a"
Jul 23 03:55:52.050386 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:52.050312    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:52.050471 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:52.050410    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xvgdl" podUID="0d412971-b53a-4379-b9ee-3f347dca0b61"
Jul 23 03:55:53.047264 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.047229    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:53.047644 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.047336    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-snqz6" podUID="78a35807-3b44-4fab-b9ea-e877402a4551"
Jul 23 03:55:53.246016 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.245990    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeReady"
Jul 23 03:55:53.246190 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.246137    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 03:55:53.281992 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.281967    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:55:53.310012 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.309986    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cn9kq"]
Jul 23 03:55:53.310432 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.310407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.314223 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.314117    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 03:55:53.314357 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.314337    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 03:55:53.314557 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.314540    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dj82d\""
Jul 23 03:55:53.314659 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.314580    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 03:55:53.320829 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.320501    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 03:55:53.331980 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.331958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:55:53.332074 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.331996    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xcmgc"]
Jul 23 03:55:53.332503 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.332480    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.335814 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.335777    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 03:55:53.335919 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.335779    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 03:55:53.335919 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.335896    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jzncz\""
Jul 23 03:55:53.350075 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.350054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xcmgc"]
Jul 23 03:55:53.350171 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.350092    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn9kq"]
Jul 23 03:55:53.350228 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.350174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.353876 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.353852    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 03:55:53.353996 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.353937    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 03:55:53.354201 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.354145    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 03:55:53.354309 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.354200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4p5f\""
Jul 23 03:55:53.424760 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.424865 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2qzd\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.424865 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqtjj\" (UniqueName: \"kubernetes.io/projected/d1646b1d-eac6-4790-a075-e03e632f44ed-kube-api-access-lqtjj\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.424865 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.425008 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.425008 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.425008 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.424912    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.425152 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.425011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1646b1d-eac6-4790-a075-e03e632f44ed-tmp-dir\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.425152 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.425048    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.425152 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.425076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.425152 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.425101    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1646b1d-eac6-4790-a075-e03e632f44ed-config-volume\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.425152 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.425137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.447851 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.447821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:53.525866 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2qzd\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.525866 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqtjj\" (UniqueName: \"kubernetes.io/projected/d1646b1d-eac6-4790-a075-e03e632f44ed-kube-api-access-lqtjj\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f65d93-ddb5-4809-a5fc-cc584a12a728-cert\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.526013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.525979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1646b1d-eac6-4790-a075-e03e632f44ed-tmp-dir\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7vr\" (UniqueName: \"kubernetes.io/projected/f4f65d93-ddb5-4809-a5fc-cc584a12a728-kube-api-access-ct7vr\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.526066    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.526129    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:55:54.026111642 +0000 UTC m=+36.487255297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1646b1d-eac6-4790-a075-e03e632f44ed-config-volume\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526681 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.526303    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:55:53.526681 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.526316    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:55:53.526681 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526681 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:53.526353    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:54.026339464 +0000 UTC m=+36.487483128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:55:53.526681 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1646b1d-eac6-4790-a075-e03e632f44ed-tmp-dir\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526932 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526726    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1646b1d-eac6-4790-a075-e03e632f44ed-config-volume\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.526932 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526850    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.526932 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.526859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.528370 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.528333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:53.530283 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.530265    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.530283 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.530278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.535328 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.535307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqtjj\" (UniqueName: \"kubernetes.io/projected/d1646b1d-eac6-4790-a075-e03e632f44ed-kube-api-access-lqtjj\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:53.535886 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.535870    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2qzd\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.536019 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.536000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:53.626871 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.626846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f65d93-ddb5-4809-a5fc-cc584a12a728-cert\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.626954 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.626915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7vr\" (UniqueName: \"kubernetes.io/projected/f4f65d93-ddb5-4809-a5fc-cc584a12a728-kube-api-access-ct7vr\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.628745 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.628722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f65d93-ddb5-4809-a5fc-cc584a12a728-cert\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.634491 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.634472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7vr\" (UniqueName: \"kubernetes.io/projected/f4f65d93-ddb5-4809-a5fc-cc584a12a728-kube-api-access-ct7vr\") pod \"ingress-canary-xcmgc\" (UID: \"f4f65d93-ddb5-4809-a5fc-cc584a12a728\") " pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.660587 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.660565    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xcmgc"
Jul 23 03:55:53.994488 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:53.994332    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xcmgc"]
Jul 23 03:55:53.998029 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:55:53.998005    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f65d93_ddb5_4809_a5fc_cc584a12a728.slice/crio-d5afa72f746df6a28d1824102fd67950aa65e72aef0de98c6ff739b96f321b79 WatchSource:0}: Error finding container d5afa72f746df6a28d1824102fd67950aa65e72aef0de98c6ff739b96f321b79: Status 404 returned error can't find the container with id d5afa72f746df6a28d1824102fd67950aa65e72aef0de98c6ff739b96f321b79
Jul 23 03:55:54.030534 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.030510    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:54.030644 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.030582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:54.030688 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:54.030672    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:55:54.030688 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:54.030684    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:55:54.030747 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:54.030698    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:55:54.030747 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:54.030743    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:55.03072975 +0000 UTC m=+37.491873404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:55:54.030840 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:54.030757    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:55:55.030751824 +0000 UTC m=+37.491895474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:55:54.047176 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.047149    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:55:54.047320 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.047151    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:55:54.049749 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.049724    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 03:55:54.049749 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.049724    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zlbwn\""
Jul 23 03:55:54.049921 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.049782    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 03:55:54.049921 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.049791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 03:55:54.337916 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.337846    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="b074e89d13aaec95113caa83e3fde920b11305922e8deb23d4046afe5352c97f" exitCode=0
Jul 23 03:55:54.338064 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.337914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"b074e89d13aaec95113caa83e3fde920b11305922e8deb23d4046afe5352c97f"}
Jul 23 03:55:54.339035 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:54.339009    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xcmgc" event={"ID":"f4f65d93-ddb5-4809-a5fc-cc584a12a728","Type":"ContainerStarted","Data":"d5afa72f746df6a28d1824102fd67950aa65e72aef0de98c6ff739b96f321b79"}
Jul 23 03:55:55.036604 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.036568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:55.036768 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.036634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:55.036768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:55.036713    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:55:55.036768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:55.036730    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:55:55.036768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:55.036740    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:55:55.036942 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:55.036788    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:55:57.036770938 +0000 UTC m=+39.497914588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:55:55.036942 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:55.036818    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:55:57.036796843 +0000 UTC m=+39.497940493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:55:55.046695 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.046677    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:55:55.049041 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.049026    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 03:55:55.049346 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.049052    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l57h5\""
Jul 23 03:55:55.344169 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.344087    2582 generic.go:356] "Generic (PLEG): container finished" podID="65fd0ded-4df0-481e-8e96-cbddc1fdb592" containerID="b775469d9b2f9d2f3839ae12ed5c27675c028b45cb47f83fe5c2ec5aeab47a43" exitCode=0
Jul 23 03:55:55.344169 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:55.344136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerDied","Data":"b775469d9b2f9d2f3839ae12ed5c27675c028b45cb47f83fe5c2ec5aeab47a43"}
Jul 23 03:55:56.165892 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.165672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:56.238550 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.238498    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:56.350643 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.350579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" event={"ID":"65fd0ded-4df0-481e-8e96-cbddc1fdb592","Type":"ContainerStarted","Data":"9b70cca29b6450d1cfd6544b0c7c3a1dbbb0459592568d2f60f5a3c1e2807ec1"}
Jul 23 03:55:56.376634 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.376387    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qk5n8" podStartSLOduration=7.051193028 podStartE2EDuration="37.376364962s" totalImagesPullingTime="30.325171934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:56.374879413 +0000 UTC m=+38.836023197" watchObservedRunningTime="2026-07-23 03:55:56.376364962 +0000 UTC m=+38.837508635"
Jul 23 03:55:56.492534 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.492511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:56.565598 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:56.565567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:57.053328 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.053298    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:55:57.053490 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.053350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:55:57.053490 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:57.053439    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:55:57.053490 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:57.053450    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:55:57.053490 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:57.053472    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:55:57.053490 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:57.053489    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:56:01.053474194 +0000 UTC m=+43.514617847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:55:57.053647 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:55:57.053513    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:01.053502111 +0000 UTC m=+43.514645782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:55:57.354717 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.354630    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xcmgc" event={"ID":"f4f65d93-ddb5-4809-a5fc-cc584a12a728","Type":"ContainerStarted","Data":"5db6e29fa278e8bfd44a10cf3755dca6c9a0ba32fa5ad616206467c83201afb7"}
Jul 23 03:55:57.372325 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.372265    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xcmgc" podStartSLOduration=1.9101566700000001 podStartE2EDuration="4.372249503s" totalImagesPullingTime="2.462092833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:55:57.371027284 +0000 UTC m=+39.832170967" watchObservedRunningTime="2026-07-23 03:55:57.372249503 +0000 UTC m=+39.833393186"
Jul 23 03:55:57.801931 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.801898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:57.902562 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:57.902527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:58.815775 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:58.815738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:58.875378 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:58.875343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:59.714658 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:59.714621    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:55:59.815354 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:55:59.815323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:00.278455 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.278424    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"]
Jul 23 03:56:00.282695 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.282680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.284862 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.284839    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 23 03:56:00.284995 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.284978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 23 03:56:00.285888 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.285873    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:56:00.285982 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.285875    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 23 03:56:00.285982 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.285903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lcxmh\""
Jul 23 03:56:00.290718 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.290696    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"]
Jul 23 03:56:00.378605 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.378581    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dddd1c9e-9491-435b-82e7-5f49299256b3-config\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.378694 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.378623    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dddd1c9e-9491-435b-82e7-5f49299256b3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.378759 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.378743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdqw9\" (UniqueName: \"kubernetes.io/projected/dddd1c9e-9491-435b-82e7-5f49299256b3-kube-api-access-sdqw9\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.384705 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.384684    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qmmfl"]
Jul 23 03:56:00.387700 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.387685    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"]
Jul 23 03:56:00.387832 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.387815    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.390682 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.390458    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 23 03:56:00.394160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.391819    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 23 03:56:00.394160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.391967    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 03:56:00.394160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.392066    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z5x89\""
Jul 23 03:56:00.394160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.392277    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 03:56:00.394443 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.394374    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.396771 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.396754    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h6fhs\""
Jul 23 03:56:00.398361 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.398341    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 23 03:56:00.398929 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.398912    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 23 03:56:00.399281 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.399265    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 03:56:00.399367 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.399348    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 03:56:00.400203 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.400188    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 23 03:56:00.403437 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.403418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"]
Jul 23 03:56:00.406403 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.406383    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qmmfl"]
Jul 23 03:56:00.478479 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.478457    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"]
Jul 23 03:56:00.479934 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.479917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dddd1c9e-9491-435b-82e7-5f49299256b3-config\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.479988 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.479944    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dddd1c9e-9491-435b-82e7-5f49299256b3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.480021 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.480011    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdqw9\" (UniqueName: \"kubernetes.io/projected/dddd1c9e-9491-435b-82e7-5f49299256b3-kube-api-access-sdqw9\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.480390 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.480375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dddd1c9e-9491-435b-82e7-5f49299256b3-config\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.481196 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.481179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.483441 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.483417    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 23 03:56:00.483544 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.483479    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4xwgt\""
Jul 23 03:56:00.483544 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.483497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 23 03:56:00.483544 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.483504    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 23 03:56:00.484325 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.484308    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dddd1c9e-9491-435b-82e7-5f49299256b3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.488471 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.488451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdqw9\" (UniqueName: \"kubernetes.io/projected/dddd1c9e-9491-435b-82e7-5f49299256b3-kube-api-access-sdqw9\") pod \"service-ca-operator-5bcbc94c78-djqdq\" (UID: \"dddd1c9e-9491-435b-82e7-5f49299256b3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.492019 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.491999    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"]
Jul 23 03:56:00.554058 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.553992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:00.580980 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.580958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5x5m\" (UniqueName: \"kubernetes.io/projected/561f167a-69fe-4e68-9c6c-638b470a31a4-kube-api-access-c5x5m\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.581050 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.580988    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.581088 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.581118 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-tmp\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.581148 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrdx\" (UniqueName: \"kubernetes.io/projected/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-kube-api-access-6xrdx\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.581180 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrh7\" (UniqueName: \"kubernetes.io/projected/6b396972-7717-4089-83b5-9f856fa2cbe9-kube-api-access-zsrh7\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.581212 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-service-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.581244 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b396972-7717-4089-83b5-9f856fa2cbe9-serving-cert\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.581275 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581240    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f167a-69fe-4e68-9c6c-638b470a31a4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.581307 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.581356 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.581300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-snapshots\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.593104 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.593087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"
Jul 23 03:56:00.609924 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.609903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:00.682366 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrh7\" (UniqueName: \"kubernetes.io/projected/6b396972-7717-4089-83b5-9f856fa2cbe9-kube-api-access-zsrh7\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682501 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-service-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682501 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b396972-7717-4089-83b5-9f856fa2cbe9-serving-cert\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682501 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f167a-69fe-4e68-9c6c-638b470a31a4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.682501 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.682702 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-snapshots\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682702 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:00.682568    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 03:56:00.682702 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5x5m\" (UniqueName: \"kubernetes.io/projected/561f167a-69fe-4e68-9c6c-638b470a31a4-kube-api-access-c5x5m\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.682702 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:00.682631    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls podName:62b6fdb3-a0ab-42de-bcfd-43c4c3738252 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:01.182610881 +0000 UTC m=+43.643754536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z5km9" (UID: "62b6fdb3-a0ab-42de-bcfd-43c4c3738252") : secret "samples-operator-tls" not found
Jul 23 03:56:00.682702 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.682974 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682974 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-tmp\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.682974 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682822    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrdx\" (UniqueName: \"kubernetes.io/projected/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-kube-api-access-6xrdx\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.683124 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.682989    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-service-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.683305 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.683282    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-snapshots\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.683381 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.683335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b396972-7717-4089-83b5-9f856fa2cbe9-tmp\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.683381 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:00.683374    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:00.683490 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:00.683425    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:01.183406831 +0000 UTC m=+43.644550481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:00.683600 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.683581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f167a-69fe-4e68-9c6c-638b470a31a4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.683818 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.683781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b396972-7717-4089-83b5-9f856fa2cbe9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.685784 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.685707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b396972-7717-4089-83b5-9f856fa2cbe9-serving-cert\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.691990 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.691961    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrdx\" (UniqueName: \"kubernetes.io/projected/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-kube-api-access-6xrdx\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:00.692098 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.691993    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5x5m\" (UniqueName: \"kubernetes.io/projected/561f167a-69fe-4e68-9c6c-638b470a31a4-kube-api-access-c5x5m\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:00.692154 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.692140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrh7\" (UniqueName: \"kubernetes.io/projected/6b396972-7717-4089-83b5-9f856fa2cbe9-kube-api-access-zsrh7\") pod \"insights-operator-555d9d7885-qmmfl\" (UID: \"6b396972-7717-4089-83b5-9f856fa2cbe9\") " pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.698101 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.698082    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-qmmfl"
Jul 23 03:56:00.714720 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.714696    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq"]
Jul 23 03:56:00.717611 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:00.717591    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddd1c9e_9491_435b_82e7_5f49299256b3.slice/crio-424beb8f44d2765664f99c36b3a8d41234802e80ccf69c1f4f0b694bd24630fb WatchSource:0}: Error finding container 424beb8f44d2765664f99c36b3a8d41234802e80ccf69c1f4f0b694bd24630fb: Status 404 returned error can't find the container with id 424beb8f44d2765664f99c36b3a8d41234802e80ccf69c1f4f0b694bd24630fb
Jul 23 03:56:00.854781 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:00.854716    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qmmfl"]
Jul 23 03:56:00.857282 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:00.857255    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b396972_7717_4089_83b5_9f856fa2cbe9.slice/crio-be5b33e5a978798546933210d5f404ff51e732dd0ee2e1c91a43624f94a9e760 WatchSource:0}: Error finding container be5b33e5a978798546933210d5f404ff51e732dd0ee2e1c91a43624f94a9e760: Status 404 returned error can't find the container with id be5b33e5a978798546933210d5f404ff51e732dd0ee2e1c91a43624f94a9e760
Jul 23 03:56:01.086349 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.086323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.086366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.086470    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.086473    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.086490    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.086518    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:56:09.086504177 +0000 UTC m=+51.547647828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:56:01.086657 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.086534    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:09.086526366 +0000 UTC m=+51.547670016 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:56:01.186828 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.186754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:01.186937 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.186826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:01.186937 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.186899    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 03:56:01.186937 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.186909    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:01.187026 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.186957    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls podName:62b6fdb3-a0ab-42de-bcfd-43c4c3738252 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:02.186938692 +0000 UTC m=+44.648082361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z5km9" (UID: "62b6fdb3-a0ab-42de-bcfd-43c4c3738252") : secret "samples-operator-tls" not found
Jul 23 03:56:01.187026 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:01.186972    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:02.186965419 +0000 UTC m=+44.648109069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:01.365739 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.365704    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qmmfl" event={"ID":"6b396972-7717-4089-83b5-9f856fa2cbe9","Type":"ContainerStarted","Data":"be5b33e5a978798546933210d5f404ff51e732dd0ee2e1c91a43624f94a9e760"}
Jul 23 03:56:01.366943 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:01.366914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq" event={"ID":"dddd1c9e-9491-435b-82e7-5f49299256b3","Type":"ContainerStarted","Data":"424beb8f44d2765664f99c36b3a8d41234802e80ccf69c1f4f0b694bd24630fb"}
Jul 23 03:56:02.143709 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.143656    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:02.182893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.182858    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"]
Jul 23 03:56:02.195464 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.195422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:02.195623 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.195548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:02.195824 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:02.195787    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:02.195904 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:02.195890    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:04.195863855 +0000 UTC m=+46.657007508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:02.200560 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:02.196333    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 03:56:02.200560 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:02.196440    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls podName:62b6fdb3-a0ab-42de-bcfd-43c4c3738252 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:04.196421254 +0000 UTC m=+46.657564918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z5km9" (UID: "62b6fdb3-a0ab-42de-bcfd-43c4c3738252") : secret "samples-operator-tls" not found
Jul 23 03:56:02.205137 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.205111    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"]
Jul 23 03:56:02.205284 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.205217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"
Jul 23 03:56:02.207645 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.207611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jtk6n\""
Jul 23 03:56:02.226483 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.226446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:02.397839 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.397746    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qq4q\" (UniqueName: \"kubernetes.io/projected/a6e53108-5ac8-478d-bb90-8d9c079e16fd-kube-api-access-2qq4q\") pod \"network-check-source-7bc5b6bf85-7r456\" (UID: \"a6e53108-5ac8-478d-bb90-8d9c079e16fd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"
Jul 23 03:56:02.498474 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.498431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qq4q\" (UniqueName: \"kubernetes.io/projected/a6e53108-5ac8-478d-bb90-8d9c079e16fd-kube-api-access-2qq4q\") pod \"network-check-source-7bc5b6bf85-7r456\" (UID: \"a6e53108-5ac8-478d-bb90-8d9c079e16fd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"
Jul 23 03:56:02.510574 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.510545    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qq4q\" (UniqueName: \"kubernetes.io/projected/a6e53108-5ac8-478d-bb90-8d9c079e16fd-kube-api-access-2qq4q\") pod \"network-check-source-7bc5b6bf85-7r456\" (UID: \"a6e53108-5ac8-478d-bb90-8d9c079e16fd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"
Jul 23 03:56:02.517241 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.517217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"
Jul 23 03:56:02.534745 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.534705    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:02.637369 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:02.637336    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:03.166318 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:03.166273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:03.266794 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:03.266758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:03.320235 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:03.320183    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:03.851539 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:03.851393    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456"]
Jul 23 03:56:03.853302 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:03.853264    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e53108_5ac8_478d_bb90_8d9c079e16fd.slice/crio-26f2c55c015e42ad9331cb3e0bb8fd9440004982327af067c020a21c26f7e265 WatchSource:0}: Error finding container 26f2c55c015e42ad9331cb3e0bb8fd9440004982327af067c020a21c26f7e265: Status 404 returned error can't find the container with id 26f2c55c015e42ad9331cb3e0bb8fd9440004982327af067c020a21c26f7e265
Jul 23 03:56:04.213434 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.213405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:04.213616 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.213459    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:04.213616 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:04.213551    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 03:56:04.213616 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:04.213576    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:04.213616 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:04.213613    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls podName:62b6fdb3-a0ab-42de-bcfd-43c4c3738252 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:08.213597969 +0000 UTC m=+50.674741619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z5km9" (UID: "62b6fdb3-a0ab-42de-bcfd-43c4c3738252") : secret "samples-operator-tls" not found
Jul 23 03:56:04.213860 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:04.213626    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:08.213620321 +0000 UTC m=+50.674763970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:04.375944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.375903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qmmfl" event={"ID":"6b396972-7717-4089-83b5-9f856fa2cbe9","Type":"ContainerStarted","Data":"f56c6482e824b8f911c88a4760cdc792b599a45fac6cfdf0d603c8aea47325a3"}
Jul 23 03:56:04.377649 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.377617    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456" event={"ID":"a6e53108-5ac8-478d-bb90-8d9c079e16fd","Type":"ContainerStarted","Data":"26f2c55c015e42ad9331cb3e0bb8fd9440004982327af067c020a21c26f7e265"}
Jul 23 03:56:04.379504 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.379430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq" event={"ID":"dddd1c9e-9491-435b-82e7-5f49299256b3","Type":"ContainerStarted","Data":"4dbf9b28f633fc97744d19348d73a65145a1ccb52ed1a1d3717d43bd12bc69af"}
Jul 23 03:56:04.395985 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.395938    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-qmmfl" podStartSLOduration=1.518920464 podStartE2EDuration="4.39591967s" totalImagesPullingTime="2.876999206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:04.395091774 +0000 UTC m=+46.856235446" watchObservedRunningTime="2026-07-23 03:56:04.39591967 +0000 UTC m=+46.857063343"
Jul 23 03:56:04.410125 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.410070    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq" podStartSLOduration=1.397764449 podStartE2EDuration="4.410052879s" totalImagesPullingTime="3.01228843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:04.409311203 +0000 UTC m=+46.870454902" watchObservedRunningTime="2026-07-23 03:56:04.410052879 +0000 UTC m=+46.871196554"
Jul 23 03:56:04.927516 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:04.927480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.169628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.169509    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-np2cq"]
Jul 23 03:56:05.193928 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.193850    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-np2cq"]
Jul 23 03:56:05.194071 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.193975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.196311 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.196286    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 03:56:05.196461 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.196382    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd7d8\""
Jul 23 03:56:05.197449 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.197429    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 03:56:05.203071 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.202261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.223911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.221252    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-data-volume\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.223911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.221319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.223911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.221391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qw4q\" (UniqueName: \"kubernetes.io/projected/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-api-access-6qw4q\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.223911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.221448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-crio-socket\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.223911 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.221498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.290185 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.290149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.321919 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.321885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-data-volume\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322062 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.321953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322062 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.321981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qw4q\" (UniqueName: \"kubernetes.io/projected/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-api-access-6qw4q\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322062 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.322021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-crio-socket\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322062 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.322056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322358 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.322128    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:05.322358 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.322193    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 03:56:05.322358 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.322201    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:05.822179606 +0000 UTC m=+48.283323260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:05.322358 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.322283    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:05.822249218 +0000 UTC m=+48.283392897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : configmap "kube-rbac-proxy" not found
Jul 23 03:56:05.322358 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.322327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-data-volume\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.322675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.322405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-crio-socket\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.332880 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.332859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qw4q\" (UniqueName: \"kubernetes.io/projected/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-api-access-6qw4q\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.390155 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.390125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.444956 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.444868    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.610832 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.610759    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.718123 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.718042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.725979 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.725954    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"]
Jul 23 03:56:05.740746 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.740547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"
Jul 23 03:56:05.741795 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.741755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"]
Jul 23 03:56:05.744077 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.744055    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zvzgg\""
Jul 23 03:56:05.744225 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.744125    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 23 03:56:05.744501 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.744483    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 23 03:56:05.811181 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.811150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.825094 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.825065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6b4r\" (UniqueName: \"kubernetes.io/projected/ec00ba70-a009-432c-8e9c-861caf24c734-kube-api-access-s6b4r\") pod \"migrator-7cc9c7ffd7-tzf8g\" (UID: \"ec00ba70-a009-432c-8e9c-861caf24c734\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"
Jul 23 03:56:05.825223 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.825172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.825292 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.825245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:05.825292 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.825273    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:05.825398 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.825329    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:06.825310895 +0000 UTC m=+49.286454548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:05.825398 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.825373    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 03:56:05.825506 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:05.825422    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:06.825408886 +0000 UTC m=+49.286552537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : configmap "kube-rbac-proxy" not found
Jul 23 03:56:05.912376 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.912339    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:05.926514 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.926482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6b4r\" (UniqueName: \"kubernetes.io/projected/ec00ba70-a009-432c-8e9c-861caf24c734-kube-api-access-s6b4r\") pod \"migrator-7cc9c7ffd7-tzf8g\" (UID: \"ec00ba70-a009-432c-8e9c-861caf24c734\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"
Jul 23 03:56:05.934926 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:05.934903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6b4r\" (UniqueName: \"kubernetes.io/projected/ec00ba70-a009-432c-8e9c-861caf24c734-kube-api-access-s6b4r\") pod \"migrator-7cc9c7ffd7-tzf8g\" (UID: \"ec00ba70-a009-432c-8e9c-861caf24c734\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"
Jul 23 03:56:06.050122 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.050094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"
Jul 23 03:56:06.075387 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.075359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:06.175494 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.175457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:06.576417 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.576390    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dd5td_71ede43b-4d5c-4ee5-87c0-384f67a7509d/dns-node-resolver/0.log"
Jul 23 03:56:06.834158 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.834120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:06.834302 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:06.834203    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 03:56:06.834302 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:06.834271    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:08.834254221 +0000 UTC m=+51.295397870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : configmap "kube-rbac-proxy" not found
Jul 23 03:56:06.834413 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.834328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:06.834472 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:06.834431    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:06.834524 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:06.834479    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:08.83446693 +0000 UTC m=+51.295610590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:06.900438 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:06.900416    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g"]
Jul 23 03:56:06.902677 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:06.902637    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec00ba70_a009_432c_8e9c_861caf24c734.slice/crio-e7facd479cecbbf559d42106f8d6450ca3108dcd480aab4b88e7771e7a14304f WatchSource:0}: Error finding container e7facd479cecbbf559d42106f8d6450ca3108dcd480aab4b88e7771e7a14304f: Status 404 returned error can't find the container with id e7facd479cecbbf559d42106f8d6450ca3108dcd480aab4b88e7771e7a14304f
Jul 23 03:56:07.388132 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.388057    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456" event={"ID":"a6e53108-5ac8-478d-bb90-8d9c079e16fd","Type":"ContainerStarted","Data":"84ce543dedc45554861dbfe5706a43b87ca845e6e80dda56b2b151c2695788cf"}
Jul 23 03:56:07.389173 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.389151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g" event={"ID":"ec00ba70-a009-432c-8e9c-861caf24c734","Type":"ContainerStarted","Data":"e7facd479cecbbf559d42106f8d6450ca3108dcd480aab4b88e7771e7a14304f"}
Jul 23 03:56:07.405425 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.405372    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7r456" podStartSLOduration=2.136587047 podStartE2EDuration="5.405356689s" totalImagesPullingTime="3.268769642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:07.404685187 +0000 UTC m=+49.865828872" watchObservedRunningTime="2026-07-23 03:56:07.405356689 +0000 UTC m=+49.866500351"
Jul 23 03:56:07.674347 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.674271    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hx47s"]
Jul 23 03:56:07.689936 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.689905    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hx47s"]
Jul 23 03:56:07.690102 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.690023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.692589 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.692560    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 23 03:56:07.693416 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.693395    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h5jfr\""
Jul 23 03:56:07.693565 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.693443    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 23 03:56:07.693643 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.693612    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 23 03:56:07.693707 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.693690    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 23 03:56:07.742531 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.742506    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-cabundle\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.742647 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.742549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5zpc\" (UniqueName: \"kubernetes.io/projected/7da2d18d-0b87-4189-850c-4f2bdabdbb79-kube-api-access-c5zpc\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.742647 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.742599    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-key\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.789551 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.789531    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nbptw_09f1bdd1-e10c-4253-8f7c-1606d3e0b58a/node-ca/0.log"
Jul 23 03:56:07.842979 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.842940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-key\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.843203 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.843184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-cabundle\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.843334 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.843227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5zpc\" (UniqueName: \"kubernetes.io/projected/7da2d18d-0b87-4189-850c-4f2bdabdbb79-kube-api-access-c5zpc\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.843997 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.843958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-cabundle\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.845904 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.845860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7da2d18d-0b87-4189-850c-4f2bdabdbb79-signing-key\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.856120 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.856096    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:07.858564 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.858540    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5zpc\" (UniqueName: \"kubernetes.io/projected/7da2d18d-0b87-4189-850c-4f2bdabdbb79-kube-api-access-c5zpc\") pod \"service-ca-58598695f7-hx47s\" (UID: \"7da2d18d-0b87-4189-850c-4f2bdabdbb79\") " pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:07.976051 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:07.975969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:08.001001 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.000980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-hx47s"
Jul 23 03:56:08.152247 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.152221    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-hx47s"]
Jul 23 03:56:08.154981 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:08.154954    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da2d18d_0b87_4189_850c_4f2bdabdbb79.slice/crio-05f85f32c5cad301a5874bd8fe7dc7f73e6d3c5a83ff03a72af4a0171d754ec6 WatchSource:0}: Error finding container 05f85f32c5cad301a5874bd8fe7dc7f73e6d3c5a83ff03a72af4a0171d754ec6: Status 404 returned error can't find the container with id 05f85f32c5cad301a5874bd8fe7dc7f73e6d3c5a83ff03a72af4a0171d754ec6
Jul 23 03:56:08.160096 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.160065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:08.247083 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.247059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:08.247215 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.247159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:08.247345 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.247323    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:08.247449 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.247382    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 03:56:08.247449 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.247398    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:16.247376486 +0000 UTC m=+58.708520153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:08.247449 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.247445    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls podName:62b6fdb3-a0ab-42de-bcfd-43c4c3738252 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:16.247426734 +0000 UTC m=+58.708570403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-z5km9" (UID: "62b6fdb3-a0ab-42de-bcfd-43c4c3738252") : secret "samples-operator-tls" not found
Jul 23 03:56:08.393425 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.393383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-hx47s" event={"ID":"7da2d18d-0b87-4189-850c-4f2bdabdbb79","Type":"ContainerStarted","Data":"05f85f32c5cad301a5874bd8fe7dc7f73e6d3c5a83ff03a72af4a0171d754ec6"}
Jul 23 03:56:08.578052 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.578028    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xcmgc_f4f65d93-ddb5-4809-a5fc-cc584a12a728/serve-healthcheck-canary/0.log"
Jul 23 03:56:08.781298 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.781270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:08.852546 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.852516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:08.852662 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:08.852563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:08.852662 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.852622    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 03:56:08.852768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.852658    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:08.852768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.852664    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:12.852647089 +0000 UTC m=+55.313790738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : configmap "kube-rbac-proxy" not found
Jul 23 03:56:08.852768 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:08.852763    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:12.852752145 +0000 UTC m=+55.313895795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:09.156138 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.156057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:09.156138 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.156105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:09.156307 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:09.156211    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 03:56:09.156307 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:09.156222    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 03:56:09.156307 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:09.156227    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74dd7447bd-4wc8g: secret "image-registry-tls" not found
Jul 23 03:56:09.156307 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:09.156278    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls podName:d1646b1d-eac6-4790-a075-e03e632f44ed nodeName:}" failed. No retries permitted until 2026-07-23 03:56:25.156264092 +0000 UTC m=+67.617407742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls") pod "dns-default-cn9kq" (UID: "d1646b1d-eac6-4790-a075-e03e632f44ed") : secret "dns-default-metrics-tls" not found
Jul 23 03:56:09.156307 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:09.156293    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls podName:ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:25.156287606 +0000 UTC m=+67.617431268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls") pod "image-registry-74dd7447bd-4wc8g" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2") : secret "image-registry-tls" not found
Jul 23 03:56:09.398295 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.398256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g" event={"ID":"ec00ba70-a009-432c-8e9c-861caf24c734","Type":"ContainerStarted","Data":"654661b50e38c9aeed2f0076d7710c0b58f86501cbb0a02324ccb68cbb58d34a"}
Jul 23 03:56:09.398690 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.398323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g" event={"ID":"ec00ba70-a009-432c-8e9c-861caf24c734","Type":"ContainerStarted","Data":"fdfe0019a5555f42b156bf7fcafb75be911950997987d7092ebc106ed85e2078"}
Jul 23 03:56:09.399756 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.399725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-hx47s" event={"ID":"7da2d18d-0b87-4189-850c-4f2bdabdbb79","Type":"ContainerStarted","Data":"ba42a7a6619ac7509a39ddd257d240a0a61a96546656cb1ba162f95b4df87ff1"}
Jul 23 03:56:09.416701 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.416617    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tzf8g" podStartSLOduration=2.579173836 podStartE2EDuration="4.416603953s" totalImagesPullingTime="1.837430117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:09.416568786 +0000 UTC m=+51.877712459" watchObservedRunningTime="2026-07-23 03:56:09.416603953 +0000 UTC m=+51.877747624"
Jul 23 03:56:09.434602 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.434561    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-hx47s" podStartSLOduration=2.434544216 podStartE2EDuration="2.434544216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:09.434196522 +0000 UTC m=+51.895340194" watchObservedRunningTime="2026-07-23 03:56:09.434544216 +0000 UTC m=+51.895687883"
Jul 23 03:56:09.917523 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:09.917487    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:10.010858 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:10.010821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:10.100546 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:10.100509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:10.268716 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:10.268685    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:10.383221 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:10.383182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:10.591567 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:10.591479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:11.203481 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:11.203424    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:11.813909 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:11.813872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:12.016837 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.016793    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:12.199679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.199610    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:12.409535 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.409503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:12.888176 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.888142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:12.888541 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.888195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:12.888541 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:12.888285    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:12.888541 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:12.888347    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls podName:a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b nodeName:}" failed. No retries permitted until 2026-07-23 03:56:20.888330946 +0000 UTC m=+63.349474597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-np2cq" (UID: "a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b") : secret "insights-runtime-extractor-tls" not found
Jul 23 03:56:12.888672 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:12.888656    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:13.012436 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:13.012401    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:13.199927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:13.199820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:13.402606 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:13.402570    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:13.796277 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:13.796241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:13.871024 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:13.870988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:14.000114 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:14.000071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:14.406722 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:14.406687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:14.482108 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:14.482078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:15.000046 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:15.000016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:15.537121 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:15.537086    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:15.632390 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:15.632357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:16.048020 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.047989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:16.109662 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.109604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:16.317547 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.317482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:16.317547 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.317539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:16.317745 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:16.317656    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:16.317745 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:56:16.317725    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls podName:561f167a-69fe-4e68-9c6c-638b470a31a4 nodeName:}" failed. No retries permitted until 2026-07-23 03:56:32.317705432 +0000 UTC m=+74.778849082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-69wkz" (UID: "561f167a-69fe-4e68-9c6c-638b470a31a4") : secret "cluster-monitoring-operator-tls" not found
Jul 23 03:56:16.319922 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.319897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/62b6fdb3-a0ab-42de-bcfd-43c4c3738252-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-z5km9\" (UID: \"62b6fdb3-a0ab-42de-bcfd-43c4c3738252\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:16.397922 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.397886    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"
Jul 23 03:56:16.398674 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.398647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:16.521085 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.521057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9"]
Jul 23 03:56:16.994918 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:16.994891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:17.405313 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.405233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:17.429768 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.429736    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9" event={"ID":"62b6fdb3-a0ab-42de-bcfd-43c4c3738252","Type":"ContainerStarted","Data":"65884a3249a4d9bb6677a349b0c72071b38506386c7744d0c75f55485765527c"}
Jul 23 03:56:17.537634 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.537592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:17.639948 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.639912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:17.794757 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.794690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:17.995242 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:17.995189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:18.331639 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:18.331612    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjk9l"
Jul 23 03:56:18.398695 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:18.398655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:19.199942 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.199904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:19.393967 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.393931    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:19.437488 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.437459    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9" event={"ID":"62b6fdb3-a0ab-42de-bcfd-43c4c3738252","Type":"ContainerStarted","Data":"bcc0b484afb97076444d8ada536680c125706484cdd3b62d8d588268c7f17c5c"}
Jul 23 03:56:19.437628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.437535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9" event={"ID":"62b6fdb3-a0ab-42de-bcfd-43c4c3738252","Type":"ContainerStarted","Data":"b4a533493c4a4d69a311caa985404dbf81663b0a58f3ab41d36797d53be9b0e1"}
Jul 23 03:56:19.454773 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.454691    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-z5km9" podStartSLOduration=17.583229572 podStartE2EDuration="19.454677536s" totalImagesPullingTime="1.871447964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:19.453625722 +0000 UTC m=+61.914769394" watchObservedRunningTime="2026-07-23 03:56:19.454677536 +0000 UTC m=+61.915821249"
Jul 23 03:56:19.593877 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.593847    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:19.719319 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.719253    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:19.999898 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:19.999870    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:20.237348 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:20.237318    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:20.405958 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:20.405889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:20.600505 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:20.600473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:20.957278 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:20.957242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:20.959457 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:20.959432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-np2cq\" (UID: \"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b\") " pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:21.108356 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:21.108329    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd7d8\""
Jul 23 03:56:21.116236 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:21.116217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-np2cq"
Jul 23 03:56:21.240688 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:21.240665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-np2cq"]
Jul 23 03:56:21.242882 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:21.242855    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17fc82a_a365_4a22_8b9a_fcbc45a3fa9b.slice/crio-f8bfac5e36dacc83825f118d65b160d70b87af1dcff6278eff5a496d3d6c8053 WatchSource:0}: Error finding container f8bfac5e36dacc83825f118d65b160d70b87af1dcff6278eff5a496d3d6c8053: Status 404 returned error can't find the container with id f8bfac5e36dacc83825f118d65b160d70b87af1dcff6278eff5a496d3d6c8053
Jul 23 03:56:21.444721 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:21.444689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-np2cq" event={"ID":"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b","Type":"ContainerStarted","Data":"f8bfac5e36dacc83825f118d65b160d70b87af1dcff6278eff5a496d3d6c8053"}
Jul 23 03:56:23.881286 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.881239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:56:23.881701 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.881333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:56:23.884204 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.884175    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 03:56:23.884808 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.884782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 03:56:23.894053 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.894029    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58af239e-1d0b-472a-8dc7-ddc26061a37a-original-pull-secret\") pod \"global-pull-secret-syncer-tzgz6\" (UID: \"58af239e-1d0b-472a-8dc7-ddc26061a37a\") " pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:56:23.894160 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.894055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78a35807-3b44-4fab-b9ea-e877402a4551-metrics-certs\") pod \"network-metrics-daemon-snqz6\" (UID: \"78a35807-3b44-4fab-b9ea-e877402a4551\") " pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:56:23.981925 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.981886    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:56:23.984581 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:23.984552    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvnt8\" (UniqueName: \"kubernetes.io/projected/0d412971-b53a-4379-b9ee-3f347dca0b61-kube-api-access-lvnt8\") pod \"network-check-target-xvgdl\" (UID: \"0d412971-b53a-4379-b9ee-3f347dca0b61\") " pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:56:24.062785 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:24.062754    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zlbwn\""
Jul 23 03:56:24.070190 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:24.070172    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:56:24.098696 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:24.098676    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tzgz6"
Jul 23 03:56:24.157577 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:24.157519    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l57h5\""
Jul 23 03:56:24.165780 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:24.165756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-snqz6"
Jul 23 03:56:25.192076 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.192043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:25.192076 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.192092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:25.194494 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.194459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1646b1d-eac6-4790-a075-e03e632f44ed-metrics-tls\") pod \"dns-default-cn9kq\" (UID: \"d1646b1d-eac6-4790-a075-e03e632f44ed\") " pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:25.194721 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.194694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"image-registry-74dd7447bd-4wc8g\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") " pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:25.425591 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.425563    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dj82d\""
Jul 23 03:56:25.433668 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.433646    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:25.444375 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.444329    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jzncz\""
Jul 23 03:56:25.452388 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:25.452370    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:26.194929 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.194906    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-snqz6"]
Jul 23 03:56:26.198156 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:26.198132    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a35807_3b44_4fab_b9ea_e877402a4551.slice/crio-164d60e9549772b0188a01eb26c95d4da290b2cb7584430a35ea3e05a21a76d0 WatchSource:0}: Error finding container 164d60e9549772b0188a01eb26c95d4da290b2cb7584430a35ea3e05a21a76d0: Status 404 returned error can't find the container with id 164d60e9549772b0188a01eb26c95d4da290b2cb7584430a35ea3e05a21a76d0
Jul 23 03:56:26.419607 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.419582    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tzgz6"]
Jul 23 03:56:26.422865 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:26.422833    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58af239e_1d0b_472a_8dc7_ddc26061a37a.slice/crio-2d5cb4c87678620ef1934a1cea412e5559ba87140366eb6698428015c5b28125 WatchSource:0}: Error finding container 2d5cb4c87678620ef1934a1cea412e5559ba87140366eb6698428015c5b28125: Status 404 returned error can't find the container with id 2d5cb4c87678620ef1934a1cea412e5559ba87140366eb6698428015c5b28125
Jul 23 03:56:26.422980 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.422878    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xvgdl"]
Jul 23 03:56:26.423971 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.423946    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn9kq"]
Jul 23 03:56:26.426979 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.426385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:56:26.430534 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:26.430504    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1646b1d_eac6_4790_a075_e03e632f44ed.slice/crio-4762968458fbee8779de6f81de69976f006ddfd3c9af2c35897a531ac5aae560 WatchSource:0}: Error finding container 4762968458fbee8779de6f81de69976f006ddfd3c9af2c35897a531ac5aae560: Status 404 returned error can't find the container with id 4762968458fbee8779de6f81de69976f006ddfd3c9af2c35897a531ac5aae560
Jul 23 03:56:26.463154 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.463133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" event={"ID":"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2","Type":"ContainerStarted","Data":"80c044594d82192daf18f191e6d431c36825346e53534c0f5a1c45eb47aaa48b"}
Jul 23 03:56:26.464620 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.464592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-np2cq" event={"ID":"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b","Type":"ContainerStarted","Data":"8fe7527ef332e8fac09577d404708c31d863de56bc116bcf923553bb8c33ac85"}
Jul 23 03:56:26.465865 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.465841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tzgz6" event={"ID":"58af239e-1d0b-472a-8dc7-ddc26061a37a","Type":"ContainerStarted","Data":"2d5cb4c87678620ef1934a1cea412e5559ba87140366eb6698428015c5b28125"}
Jul 23 03:56:26.467005 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.466984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-snqz6" event={"ID":"78a35807-3b44-4fab-b9ea-e877402a4551","Type":"ContainerStarted","Data":"164d60e9549772b0188a01eb26c95d4da290b2cb7584430a35ea3e05a21a76d0"}
Jul 23 03:56:26.468100 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.468083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xvgdl" event={"ID":"0d412971-b53a-4379-b9ee-3f347dca0b61","Type":"ContainerStarted","Data":"17b597b29c084b72d5c0d17833d2c2026fe5ed8cedffede40ded29debc4b4bed"}
Jul 23 03:56:26.468985 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:26.468967    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9kq" event={"ID":"d1646b1d-eac6-4790-a075-e03e632f44ed","Type":"ContainerStarted","Data":"4762968458fbee8779de6f81de69976f006ddfd3c9af2c35897a531ac5aae560"}
Jul 23 03:56:27.478947 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.478903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xvgdl" event={"ID":"0d412971-b53a-4379-b9ee-3f347dca0b61","Type":"ContainerStarted","Data":"90d4ad7aed46d8e0e19fcca149ffcb93d7147d51168228e205f1a95fcbef9089"}
Jul 23 03:56:27.479410 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.479195    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:56:27.482508 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.482480    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" event={"ID":"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2","Type":"ContainerStarted","Data":"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"}
Jul 23 03:56:27.482694 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.482680    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:27.498976 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.498211    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xvgdl" podStartSLOduration=68.498193889 podStartE2EDuration="1m8.498193889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:27.496910021 +0000 UTC m=+69.958053718" watchObservedRunningTime="2026-07-23 03:56:27.498193889 +0000 UTC m=+69.959337562"
Jul 23 03:56:27.520463 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:27.520410    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" podStartSLOduration=68.520392335 podStartE2EDuration="1m8.520392335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:27.520133412 +0000 UTC m=+69.981277111" watchObservedRunningTime="2026-07-23 03:56:27.520392335 +0000 UTC m=+69.981536008"
Jul 23 03:56:28.488224 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:28.488187    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-snqz6" event={"ID":"78a35807-3b44-4fab-b9ea-e877402a4551","Type":"ContainerStarted","Data":"49a5fc446ae548a10acf3d81dacb749b008b5123ecdf4c95358943520c122b37"}
Jul 23 03:56:29.411337 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:29.411295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:29.509116 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:29.509076    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:29.612137 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:29.612102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:29.709010 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:29.708929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:30.859017 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:30.858975    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:30.923519 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:30.923480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:32.356929 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.356897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:32.359171 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.359132    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f167a-69fe-4e68-9c6c-638b470a31a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-69wkz\" (UID: \"561f167a-69fe-4e68-9c6c-638b470a31a4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:32.504215 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.504173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-snqz6" event={"ID":"78a35807-3b44-4fab-b9ea-e877402a4551","Type":"ContainerStarted","Data":"cc2c86469117e494f462e2ff8db3a48e44d1f46bc61df8cd6178a5abc67ebb1f"}
Jul 23 03:56:32.506013 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.505981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9kq" event={"ID":"d1646b1d-eac6-4790-a075-e03e632f44ed","Type":"ContainerStarted","Data":"028d220ae8f841d7040968199f48fde7cd6441d644c75f4117679373b4d2acb3"}
Jul 23 03:56:32.506142 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.506034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9kq" event={"ID":"d1646b1d-eac6-4790-a075-e03e632f44ed","Type":"ContainerStarted","Data":"13f9a677f242434196729eda2321853a586997b9d609844517deacc1ee2a0765"}
Jul 23 03:56:32.506142 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.506067    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:32.506673 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.506649    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h6fhs\""
Jul 23 03:56:32.507728 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.507705    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-np2cq" event={"ID":"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b","Type":"ContainerStarted","Data":"7320c32a43acab6e7a0a2ee62ae368bde6cb64160388f01ebdb4ea367e515022"}
Jul 23 03:56:32.509105 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.509083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tzgz6" event={"ID":"58af239e-1d0b-472a-8dc7-ddc26061a37a","Type":"ContainerStarted","Data":"55f180dd7eed865b156e7c8a27c5eeb0a8f9c022149a592c84c9426313665b7d"}
Jul 23 03:56:32.514369 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.514353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"
Jul 23 03:56:32.522426 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.522383    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-snqz6" podStartSLOduration=71.76393653 podStartE2EDuration="1m13.522369236s" totalImagesPullingTime="1.758432706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:32.521239911 +0000 UTC m=+74.982383584" watchObservedRunningTime="2026-07-23 03:56:32.522369236 +0000 UTC m=+74.983512944"
Jul 23 03:56:32.537328 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.537306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:32.542422 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.542385    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tzgz6" podStartSLOduration=67.802946992 podStartE2EDuration="1m13.542371841s" totalImagesPullingTime="5.739424849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:32.541793032 +0000 UTC m=+75.002936704" watchObservedRunningTime="2026-07-23 03:56:32.542371841 +0000 UTC m=+75.003515512"
Jul 23 03:56:32.564569 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.563682    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cn9kq" podStartSLOduration=33.841177886 podStartE2EDuration="39.563666551s" totalImagesPullingTime="5.722488665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:32.561996173 +0000 UTC m=+75.023139844" watchObservedRunningTime="2026-07-23 03:56:32.563666551 +0000 UTC m=+75.024810227"
Jul 23 03:56:32.627154 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.627125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:32.664361 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.664334    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz"]
Jul 23 03:56:32.666051 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:32.666027    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561f167a_69fe_4e68_9c6c_638b470a31a4.slice/crio-3fde17e9688bf999948c6861f78c22b1fd8161e81de24009ac059e6179118507 WatchSource:0}: Error finding container 3fde17e9688bf999948c6861f78c22b1fd8161e81de24009ac059e6179118507: Status 404 returned error can't find the container with id 3fde17e9688bf999948c6861f78c22b1fd8161e81de24009ac059e6179118507
Jul 23 03:56:32.727638 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:32.727602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:33.514186 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:33.514104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz" event={"ID":"561f167a-69fe-4e68-9c6c-638b470a31a4","Type":"ContainerStarted","Data":"3fde17e9688bf999948c6861f78c22b1fd8161e81de24009ac059e6179118507"}
Jul 23 03:56:36.489793 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.489756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:36.533478 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.533449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-np2cq" event={"ID":"a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b","Type":"ContainerStarted","Data":"5a87b7641ff3b6e208c1c87d395b6117ac2476ca254666ffb3f17bc7d70473f5"}
Jul 23 03:56:36.534941 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.534917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz" event={"ID":"561f167a-69fe-4e68-9c6c-638b470a31a4","Type":"ContainerStarted","Data":"9d9f78520572caec0818217eb15ffba586d5994b0c3155b632e4aedb1754af27"}
Jul 23 03:56:36.553106 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.553061    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-np2cq" podStartSLOduration=16.805575018 podStartE2EDuration="31.553046931s" totalImagesPullingTime="14.747471913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 03:56:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:36.551278998 +0000 UTC m=+79.012422681" watchObservedRunningTime="2026-07-23 03:56:36.553046931 +0000 UTC m=+79.014190954"
Jul 23 03:56:36.563727 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.563704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:36.572914 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.572871    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-69wkz" podStartSLOduration=33.078607178 podStartE2EDuration="36.572854104s" totalImagesPullingTime="3.494246926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:36.571507262 +0000 UTC m=+79.032650936" watchObservedRunningTime="2026-07-23 03:56:36.572854104 +0000 UTC m=+79.033997774"
Jul 23 03:56:36.658014 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.657986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:36.786493 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:36.786426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:39.687403 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:39.687368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:39.763262 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:39.763229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:40.213985 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:40.213943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:40.275029 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:40.274903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:40.357613 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:40.357578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:40.417464 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:40.417430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:42.516131 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:42.516102    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cn9kq"
Jul 23 03:56:42.807669 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:42.807588    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:44.851371 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:44.851337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.093719 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.093682    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.143070 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.142996    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r59j8"]
Jul 23 03:56:45.147118 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.147095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.150288 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.150266    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 03:56:45.154824 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.150510    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 03:56:45.157685 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.156870    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 03:56:45.157831 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.156945    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9z7rs\""
Jul 23 03:56:45.157944 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.157041    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 03:56:45.201918 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.199994    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.256490 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgj8f\" (UniqueName: \"kubernetes.io/projected/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-kube-api-access-vgj8f\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256622 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-wtmp\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256622 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-root\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256622 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256622 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256601    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256856 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-textfile\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256856 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-tls\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256856 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-sys\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.256856 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.256718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-metrics-client-ca\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.293629 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.293603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.357973 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.357945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-tls\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358073 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.357979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-sys\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358073 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.357996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-metrics-client-ca\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358073 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgj8f\" (UniqueName: \"kubernetes.io/projected/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-kube-api-access-vgj8f\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358073 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358060    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-wtmp\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358266 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-sys\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358266 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-root\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358266 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358266 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358266 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-textfile\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358523 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-wtmp\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358523 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-root\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358523 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-textfile\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358672 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-metrics-client-ca\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.358891 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.358871    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.360608 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.360579    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.360817 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.360788    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-node-exporter-tls\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.369741 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.369715    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgj8f\" (UniqueName: \"kubernetes.io/projected/3a25a26f-41c2-4540-9dac-5e6d1a32b1ec-kube-api-access-vgj8f\") pod \"node-exporter-r59j8\" (UID: \"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec\") " pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.392142 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.392103    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.439662 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.439589    2582 patch_prober.go:28] interesting pod/image-registry-74dd7447bd-4wc8g container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 03:56:45.439662 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.439636    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 03:56:45.462465 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.462448    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r59j8"
Jul 23 03:56:45.473459 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:45.473433    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a25a26f_41c2_4540_9dac_5e6d1a32b1ec.slice/crio-4ebf3f24471973f4a731760414504364e75680a71bad7c4bffc1587d27192d2a WatchSource:0}: Error finding container 4ebf3f24471973f4a731760414504364e75680a71bad7c4bffc1587d27192d2a: Status 404 returned error can't find the container with id 4ebf3f24471973f4a731760414504364e75680a71bad7c4bffc1587d27192d2a
Jul 23 03:56:45.494541 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.494516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.565826 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.565791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r59j8" event={"ID":"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec","Type":"ContainerStarted","Data":"4ebf3f24471973f4a731760414504364e75680a71bad7c4bffc1587d27192d2a"}
Jul 23 03:56:45.567930 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.567904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:45.676023 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:45.675969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:46.428707 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:46.428676    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:46.524046 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:46.524015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:46.570954 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:46.570891    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a25a26f-41c2-4540-9dac-5e6d1a32b1ec" containerID="867216091b7f528c9fdb00e3034021bdd3a4beef5f400c7b8b31bcf9f5186de5" exitCode=0
Jul 23 03:56:46.571060 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:46.570987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r59j8" event={"ID":"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec","Type":"ContainerDied","Data":"867216091b7f528c9fdb00e3034021bdd3a4beef5f400c7b8b31bcf9f5186de5"}
Jul 23 03:56:46.652299 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:46.652271    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:47.219960 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.219923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:47.328540 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.328507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:47.461350 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.461305    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:47.576034 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.576002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r59j8" event={"ID":"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec","Type":"ContainerStarted","Data":"6e3be54ae746d06021c23b85f2f18c41d54fa9acef59b7d1afd6acc29c3065ed"}
Jul 23 03:56:47.576178 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.576057    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r59j8" event={"ID":"3a25a26f-41c2-4540-9dac-5e6d1a32b1ec","Type":"ContainerStarted","Data":"f66e18fca2aba3194c5f4c4d2796d31d157983e39670c1b8dedca85c02ac9aec"}
Jul 23 03:56:47.588445 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.588414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:47.613543 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.613503    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r59j8" podStartSLOduration=1.740034535 podStartE2EDuration="2.613489437s" totalImagesPullingTime="810.715612ms" totalInitContainerRuntime="62.73929ms" isStatefulPod=true podCreationTimestamp="2026-07-23 03:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:56:47.613139616 +0000 UTC m=+90.074283288" watchObservedRunningTime="2026-07-23 03:56:47.613489437 +0000 UTC m=+90.074633109"
Jul 23 03:56:47.703625 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:47.703598    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:48.171173 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.171122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:48.272311 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.272275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:48.492674 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.492650    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:56:48.567752 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.567721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:48.823760 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.823681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:48.888504 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:48.888471    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:49.878400 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:49.878366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:50.423434 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:50.423398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:51.433413 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:51.433371    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:51.683072 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:51.683041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:51.764497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:51.764466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:54.320907 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.320874    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-pl647"]
Jul 23 03:56:54.325283 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.325266    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:56:54.327747 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.327724    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 03:56:54.327860 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.327724    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tj5nd\""
Jul 23 03:56:54.327928 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.327874    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 03:56:54.333878 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.333847    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pl647"]
Jul 23 03:56:54.341717 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.341694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:54.421630 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.421605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwqk\" (UniqueName: \"kubernetes.io/projected/4d6a1218-fdb1-479a-b701-dfa362d94abc-kube-api-access-9zwqk\") pod \"downloads-5bc67dcd46-pl647\" (UID: \"4d6a1218-fdb1-479a-b701-dfa362d94abc\") " pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:56:54.440225 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.440201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:54.522644 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.522618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwqk\" (UniqueName: \"kubernetes.io/projected/4d6a1218-fdb1-479a-b701-dfa362d94abc-kube-api-access-9zwqk\") pod \"downloads-5bc67dcd46-pl647\" (UID: \"4d6a1218-fdb1-479a-b701-dfa362d94abc\") " pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:56:54.531161 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.531135    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwqk\" (UniqueName: \"kubernetes.io/projected/4d6a1218-fdb1-479a-b701-dfa362d94abc-kube-api-access-9zwqk\") pod \"downloads-5bc67dcd46-pl647\" (UID: \"4d6a1218-fdb1-479a-b701-dfa362d94abc\") " pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:56:54.541183 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.541158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:54.634927 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.634871    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:56:54.765060 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.765032    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pl647"]
Jul 23 03:56:54.767989 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:56:54.767958    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6a1218_fdb1_479a_b701_dfa362d94abc.slice/crio-2ab4694bb58856de72af3561491245f735afa51a1380d9b06fe9ac6b7836ef1c WatchSource:0}: Error finding container 2ab4694bb58856de72af3561491245f735afa51a1380d9b06fe9ac6b7836ef1c: Status 404 returned error can't find the container with id 2ab4694bb58856de72af3561491245f735afa51a1380d9b06fe9ac6b7836ef1c
Jul 23 03:56:54.845602 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:54.845575    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.011851 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.011793    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.070127 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.070103    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.218968 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.218932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.286679 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.286598    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.603148 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.603071    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pl647" event={"ID":"4d6a1218-fdb1-479a-b701-dfa362d94abc","Type":"ContainerStarted","Data":"2ab4694bb58856de72af3561491245f735afa51a1380d9b06fe9ac6b7836ef1c"}
Jul 23 03:56:55.834405 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.834375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:55.897486 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.897401    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:56:55.926163 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:55.926131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:56.056645 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:56.056612    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:56.162887 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:56.162795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:56.254443 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:56.254413    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:56.456469 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:56.456381    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:57.030231 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:57.030182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:56:58.491345 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:56:58.491246    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xvgdl"
Jul 23 03:57:00.659393 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:00.659353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:00.727121 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:00.727074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:01.823695 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:01.823653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:01.923396 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:01.923349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:02.529573 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:02.529537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:02.591044 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:02.591006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:07.839249 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:07.839213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:07.938822 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:07.938763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:08.636182 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:08.636128    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:08.736663 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:08.736612    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:10.904322 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:10.904289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:11.004972 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.004935    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:11.669144 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.669106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pl647" event={"ID":"4d6a1218-fdb1-479a-b701-dfa362d94abc","Type":"ContainerStarted","Data":"194a1726c212c9bd18d328b9d19a0122673980dbf6973186205580862747c289"}
Jul 23 03:57:11.669334 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.669312    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:57:11.687854 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.687788    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-pl647" podStartSLOduration=1.539512696 podStartE2EDuration="17.687768652s" totalImagesPullingTime="16.148255956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:57:11.686252054 +0000 UTC m=+114.147395747" watchObservedRunningTime="2026-07-23 03:57:11.687768652 +0000 UTC m=+114.148912326"
Jul 23 03:57:11.691225 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.691203    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-pl647"
Jul 23 03:57:11.928290 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:11.928210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:12.028867 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.028828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:12.131329 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.131289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:12.187545 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.187455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:12.636897 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.636864    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:57:12.641465 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.641438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.644194 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.644169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hgztq\""
Jul 23 03:57:12.644360 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.644170    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 03:57:12.644448 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.644171    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 03:57:12.644448 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.644189    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 03:57:12.645282 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.645249    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 03:57:12.645282 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.645260    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 03:57:12.650491 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.650473    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 03:57:12.654023 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.653999    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:57:12.666094 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666248 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666248 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz5cf\" (UniqueName: \"kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666355 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666268    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666355 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666473 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.666526 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.666486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767503 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767689 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767689 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767598    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767689 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767689 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767689 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.767993 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.767713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz5cf\" (UniqueName: \"kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.768983 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.768381    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.768983 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.768504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.768983 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.768507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.768983 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.768935    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.770770 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.770744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.770917 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.770755    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.776465 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.776443    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz5cf\" (UniqueName: \"kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf\") pod \"console-679fb7bfb6-967v2\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") " pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:12.954276 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:12.954175    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:13.109826 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:13.109774    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:57:13.113475 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:57:13.113434    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8522954_81a0_4311_bbe3_683308e25c0d.slice/crio-1f22b0109cfabe5e660f93c1914ca9b109e72cedf1dbc35a3cc8e41fdc7cbdcc WatchSource:0}: Error finding container 1f22b0109cfabe5e660f93c1914ca9b109e72cedf1dbc35a3cc8e41fdc7cbdcc: Status 404 returned error can't find the container with id 1f22b0109cfabe5e660f93c1914ca9b109e72cedf1dbc35a3cc8e41fdc7cbdcc
Jul 23 03:57:13.679443 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:13.679400    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679fb7bfb6-967v2" event={"ID":"f8522954-81a0-4311-bbe3-683308e25c0d","Type":"ContainerStarted","Data":"1f22b0109cfabe5e660f93c1914ca9b109e72cedf1dbc35a3cc8e41fdc7cbdcc"}
Jul 23 03:57:17.544371 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:17.544328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:17.654875 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:17.654837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:17.704628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:17.704590    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679fb7bfb6-967v2" event={"ID":"f8522954-81a0-4311-bbe3-683308e25c0d","Type":"ContainerStarted","Data":"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"}
Jul 23 03:57:17.727499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:17.727444    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-679fb7bfb6-967v2" podStartSLOduration=1.991453674 podStartE2EDuration="5.72742988s" totalImagesPullingTime="3.735976206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:57:17.725637666 +0000 UTC m=+120.186781338" watchObservedRunningTime="2026-07-23 03:57:17.72742988 +0000 UTC m=+120.188573552"
Jul 23 03:57:18.446819 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:18.446777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:18.510746 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:18.510703    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:20.066860 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:20.066821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:20.139018 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:20.138980    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:20.921933 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:20.921880    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerName="registry" containerID="cri-o://4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059" gracePeriod=30
Jul 23 03:57:21.191682 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.191624    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:57:21.238396 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238511 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238420    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238511 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238453    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238511 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238483    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238655 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238509    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238655 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2qzd\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238655 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238559    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.238655 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.238603    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token\") pod \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\" (UID: \"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2\") "
Jul 23 03:57:21.239036 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.239006    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:57:21.239148 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.239101    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:57:21.241124 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.241077    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:57:21.241226 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.241185    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:57:21.241291 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.241239    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:57:21.241345 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.241308    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:57:21.241400 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.241374    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd" (OuterVolumeSpecName: "kube-api-access-r2qzd") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "kube-api-access-r2qzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:57:21.246984 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.246965    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" (UID: "ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 03:57:21.339504 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339477    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-certificates\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339504 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339502    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-installation-pull-secrets\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339516    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-trusted-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339528    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2qzd\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-kube-api-access-r2qzd\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339540    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-registry-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339552    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-bound-sa-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339563    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-ca-trust-extracted\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.339640 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.339576    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2-image-registry-private-configuration\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:57:21.720716 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.720681    2582 generic.go:356] "Generic (PLEG): container finished" podID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerID="4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059" exitCode=0
Jul 23 03:57:21.720893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.720727    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g"
Jul 23 03:57:21.720893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.720768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" event={"ID":"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2","Type":"ContainerDied","Data":"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"}
Jul 23 03:57:21.720893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.720834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74dd7447bd-4wc8g" event={"ID":"ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2","Type":"ContainerDied","Data":"80c044594d82192daf18f191e6d431c36825346e53534c0f5a1c45eb47aaa48b"}
Jul 23 03:57:21.720893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.720862    2582 scope.go:117] "RemoveContainer" containerID="4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"
Jul 23 03:57:21.729435 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.729416    2582 scope.go:117] "RemoveContainer" containerID="4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"
Jul 23 03:57:21.729685 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:57:21.729666    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059\": container with ID starting with 4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059 not found: ID does not exist" containerID="4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"
Jul 23 03:57:21.729734 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.729695    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059"} err="failed to get container status \"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059\": rpc error: code = NotFound desc = could not find container \"4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059\": container with ID starting with 4c793289887e3b439d84aab16c3b69be95b5fb5852c1b314de77d3914daf2059 not found: ID does not exist"
Jul 23 03:57:21.742629 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.742599    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:57:21.747286 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:21.747266    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74dd7447bd-4wc8g"]
Jul 23 03:57:22.050906 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:22.050881    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" path="/var/lib/kubelet/pods/ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2/volumes"
Jul 23 03:57:22.955082 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:22.955047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:22.955497 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:22.955101    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:22.959550 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:22.959531    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:23.731244 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:23.731219    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:57:23.877324 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:23.877286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:23.935422 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:23.935390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:23.999499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:23.999429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:24.737042 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:24.737009    2582 generic.go:356] "Generic (PLEG): container finished" podID="6b396972-7717-4089-83b5-9f856fa2cbe9" containerID="f56c6482e824b8f911c88a4760cdc792b599a45fac6cfdf0d603c8aea47325a3" exitCode=0
Jul 23 03:57:24.737173 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:24.737084    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qmmfl" event={"ID":"6b396972-7717-4089-83b5-9f856fa2cbe9","Type":"ContainerDied","Data":"f56c6482e824b8f911c88a4760cdc792b599a45fac6cfdf0d603c8aea47325a3"}
Jul 23 03:57:24.737531 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:24.737515    2582 scope.go:117] "RemoveContainer" containerID="f56c6482e824b8f911c88a4760cdc792b599a45fac6cfdf0d603c8aea47325a3"
Jul 23 03:57:25.742539 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:25.742504    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qmmfl" event={"ID":"6b396972-7717-4089-83b5-9f856fa2cbe9","Type":"ContainerStarted","Data":"21dd23683c083cc3fca562ddc7273eafbcbb344ef0dbf189f3957b834543d944"}
Jul 23 03:57:32.536881 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:32.536845    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:32.601584 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:32.601555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:34.772965 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:34.772930    2582 generic.go:356] "Generic (PLEG): container finished" podID="dddd1c9e-9491-435b-82e7-5f49299256b3" containerID="4dbf9b28f633fc97744d19348d73a65145a1ccb52ed1a1d3717d43bd12bc69af" exitCode=0
Jul 23 03:57:34.773319 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:34.773001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq" event={"ID":"dddd1c9e-9491-435b-82e7-5f49299256b3","Type":"ContainerDied","Data":"4dbf9b28f633fc97744d19348d73a65145a1ccb52ed1a1d3717d43bd12bc69af"}
Jul 23 03:57:34.773319 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:34.773305    2582 scope.go:117] "RemoveContainer" containerID="4dbf9b28f633fc97744d19348d73a65145a1ccb52ed1a1d3717d43bd12bc69af"
Jul 23 03:57:35.779070 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:35.779040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-djqdq" event={"ID":"dddd1c9e-9491-435b-82e7-5f49299256b3","Type":"ContainerStarted","Data":"030bd9987db8fec9d329191951a301952daf428c246642573e41ed9c7a9be5a2"}
Jul 23 03:57:51.883078 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:51.883033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:51.954642 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:51.954613    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:57:58.241166 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:57:58.241133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:01.383978 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.383937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:01.480691 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.480660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:01.731335 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.731310    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 03:58:01.731580 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.731559    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerName="registry"
Jul 23 03:58:01.731580 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.731573    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerName="registry"
Jul 23 03:58:01.731657 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.731626    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1224ee-31ef-4d30-a2d5-9ddd1c0fd2c2" containerName="registry"
Jul 23 03:58:01.734444 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.734429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.760245 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.760216    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 03:58:01.812344 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812499 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcrdw\" (UniqueName: \"kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.812675 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.812542    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.828087 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.828053    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:01.913023 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.912988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913231 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcrdw\" (UniqueName: \"kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913231 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913231 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913231 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913185    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913469 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913469 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.913469 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:01.914044 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.913988    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.914044 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.914037    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.914204 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.914106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.914204 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.914109    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.915615 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.915597    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.915788 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.915770    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:01.922573 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:01.922551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcrdw\" (UniqueName: \"kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw\") pod \"console-75665c68bb-42v6x\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") " pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:02.043636 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:02.043539    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:02.186964 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:02.186935    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 03:58:02.188338 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:58:02.188314    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e852e92_7a07_4c2d_9152_d13ece47318d.slice/crio-0424df4530bea9d3848604b4652a8f28be040d88b834b913945404e3dea1dcb3 WatchSource:0}: Error finding container 0424df4530bea9d3848604b4652a8f28be040d88b834b913945404e3dea1dcb3: Status 404 returned error can't find the container with id 0424df4530bea9d3848604b4652a8f28be040d88b834b913945404e3dea1dcb3
Jul 23 03:58:02.878636 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:02.878602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75665c68bb-42v6x" event={"ID":"6e852e92-7a07-4c2d-9152-d13ece47318d","Type":"ContainerStarted","Data":"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"}
Jul 23 03:58:02.879052 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:02.878647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75665c68bb-42v6x" event={"ID":"6e852e92-7a07-4c2d-9152-d13ece47318d","Type":"ContainerStarted","Data":"0424df4530bea9d3848604b4652a8f28be040d88b834b913945404e3dea1dcb3"}
Jul 23 03:58:02.987440 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:02.987373    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75665c68bb-42v6x" podStartSLOduration=1.987356629 podStartE2EDuration="1.987356629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:58:02.986850199 +0000 UTC m=+165.447993895" watchObservedRunningTime="2026-07-23 03:58:02.987356629 +0000 UTC m=+165.448500383"
Jul 23 03:58:12.044676 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:12.044590    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:12.044676 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:12.044661    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:12.051369 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:12.051350    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:12.916496 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:12.916465    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 03:58:12.965450 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:12.965422    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:58:13.113798 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:13.113769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:13.214661 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:13.214586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:13.315153 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:13.315121    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:13.380728 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:13.380701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:17.544783 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:17.544742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:17.645390 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:17.645361    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:18.605312 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:18.605280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:19.406758 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:19.406719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:19.499005 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:19.498971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:37.984985 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:37.984942    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-679fb7bfb6-967v2" podUID="f8522954-81a0-4311-bbe3-683308e25c0d" containerName="console" containerID="cri-o://86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b" gracePeriod=15
Jul 23 03:58:38.216822 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.216780    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:58:38.362913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.362833    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.362913 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.362871    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363116 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.362920    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363116 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.362964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363116 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363081    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz5cf\" (UniqueName: \"kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363271 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363124    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363271 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363173    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca\") pod \"f8522954-81a0-4311-bbe3-683308e25c0d\" (UID: \"f8522954-81a0-4311-bbe3-683308e25c0d\") "
Jul 23 03:58:38.363370 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363332    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config" (OuterVolumeSpecName: "console-config") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:38.363370 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363356    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:38.363527 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363504    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-trusted-ca-bundle\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.363527 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363529    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-console-config\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.363704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363607    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:38.363704 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.363646    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca" (OuterVolumeSpecName: "service-ca") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 03:58:38.365161 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.365136    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:38.365247 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.365153    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 03:58:38.365247 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.365197    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf" (OuterVolumeSpecName: "kube-api-access-wz5cf") pod "f8522954-81a0-4311-bbe3-683308e25c0d" (UID: "f8522954-81a0-4311-bbe3-683308e25c0d"). InnerVolumeSpecName "kube-api-access-wz5cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 03:58:38.463904 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.463879    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.463904 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.463899    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-oauth-config\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.464027 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.463908    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8522954-81a0-4311-bbe3-683308e25c0d-console-serving-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.464027 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.463917    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz5cf\" (UniqueName: \"kubernetes.io/projected/f8522954-81a0-4311-bbe3-683308e25c0d-kube-api-access-wz5cf\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:38.464027 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:38.463927    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8522954-81a0-4311-bbe3-683308e25c0d-oauth-serving-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 03:58:39.001628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.001593    2582 generic.go:356] "Generic (PLEG): container finished" podID="f8522954-81a0-4311-bbe3-683308e25c0d" containerID="86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b" exitCode=0
Jul 23 03:58:39.001628 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.001630    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679fb7bfb6-967v2" event={"ID":"f8522954-81a0-4311-bbe3-683308e25c0d","Type":"ContainerDied","Data":"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"}
Jul 23 03:58:39.002136 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.001650    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679fb7bfb6-967v2"
Jul 23 03:58:39.002136 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.001693    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679fb7bfb6-967v2" event={"ID":"f8522954-81a0-4311-bbe3-683308e25c0d","Type":"ContainerDied","Data":"1f22b0109cfabe5e660f93c1914ca9b109e72cedf1dbc35a3cc8e41fdc7cbdcc"}
Jul 23 03:58:39.002136 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.001712    2582 scope.go:117] "RemoveContainer" containerID="86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"
Jul 23 03:58:39.010394 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.010377    2582 scope.go:117] "RemoveContainer" containerID="86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"
Jul 23 03:58:39.010641 ip-10-0-138-244 kubenswrapper[2582]: E0723 03:58:39.010624    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b\": container with ID starting with 86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b not found: ID does not exist" containerID="86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"
Jul 23 03:58:39.010684 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.010648    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b"} err="failed to get container status \"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b\": rpc error: code = NotFound desc = could not find container \"86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b\": container with ID starting with 86bbf5ea06a2bb5f87fc099543efb5089d96fa96c59395da5ed771be41a9273b not found: ID does not exist"
Jul 23 03:58:39.026174 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.026153    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:58:39.032093 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:39.032074    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-679fb7bfb6-967v2"]
Jul 23 03:58:40.051072 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:40.051039    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8522954-81a0-4311-bbe3-683308e25c0d" path="/var/lib/kubelet/pods/f8522954-81a0-4311-bbe3-683308e25c0d/volumes"
Jul 23 03:58:49.344305 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:49.344259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:49.440747 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:49.440713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:56.122108 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:56.122077    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:58:56.213167 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:58:56.213126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:01.357602 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.357567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:01.403537 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.403503    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"]
Jul 23 03:59:01.403772 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.403760    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8522954-81a0-4311-bbe3-683308e25c0d" containerName="console"
Jul 23 03:59:01.403836 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.403773    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8522954-81a0-4311-bbe3-683308e25c0d" containerName="console"
Jul 23 03:59:01.403872 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.403841    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8522954-81a0-4311-bbe3-683308e25c0d" containerName="console"
Jul 23 03:59:01.406738 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.406721    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.414149 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.414128    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 03:59:01.414570 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.414556    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 03:59:01.414953 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.414941    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dzzlf\""
Jul 23 03:59:01.417047 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.417030    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 03:59:01.428090 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.428072    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 03:59:01.429238 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.429217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"]
Jul 23 03:59:01.429339 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.429282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8de06117-90a8-4b8f-9966-b6e6ccfe164d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.429339 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.429307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njqwg\" (UniqueName: \"kubernetes.io/projected/8de06117-90a8-4b8f-9966-b6e6ccfe164d-kube-api-access-njqwg\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.489928 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.489892    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:01.530538 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.530505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8de06117-90a8-4b8f-9966-b6e6ccfe164d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.530538 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.530542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njqwg\" (UniqueName: \"kubernetes.io/projected/8de06117-90a8-4b8f-9966-b6e6ccfe164d-kube-api-access-njqwg\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.533014 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.532987    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8de06117-90a8-4b8f-9966-b6e6ccfe164d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.553285 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.553259    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njqwg\" (UniqueName: \"kubernetes.io/projected/8de06117-90a8-4b8f-9966-b6e6ccfe164d-kube-api-access-njqwg\") pod \"managed-serviceaccount-addon-agent-59d6547794-cd654\" (UID: \"8de06117-90a8-4b8f-9966-b6e6ccfe164d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.578560 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.578530    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"]
Jul 23 03:59:01.581745 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.581723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.586039 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.586016    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 23 03:59:01.586165 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.586052    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 23 03:59:01.586165 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.586060    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 23 03:59:01.595520 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.595503    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 23 03:59:01.618025 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.617957    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"]
Jul 23 03:59:01.631089 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.631226 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.631226 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631192    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5197743-a5df-4c41-ba55-7dda189951f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.631320 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q6vc\" (UniqueName: \"kubernetes.io/projected/a5197743-a5df-4c41-ba55-7dda189951f2-kube-api-access-7q6vc\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.631320 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.631320 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.631276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.731514 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731494    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"
Jul 23 03:59:01.731654 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.731697 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5197743-a5df-4c41-ba55-7dda189951f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.731731 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q6vc\" (UniqueName: \"kubernetes.io/projected/a5197743-a5df-4c41-ba55-7dda189951f2-kube-api-access-7q6vc\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.731764 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.731764 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.731755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.732030 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.732009    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.732432 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.732406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5197743-a5df-4c41-ba55-7dda189951f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.734465 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.734443    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.734577 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.734473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.734577 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.734486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-ca\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.734577 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.734539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a5197743-a5df-4c41-ba55-7dda189951f2-hub\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.741167 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.741145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q6vc\" (UniqueName: \"kubernetes.io/projected/a5197743-a5df-4c41-ba55-7dda189951f2-kube-api-access-7q6vc\") pod \"cluster-proxy-proxy-agent-5c55996678-gbrrt\" (UID: \"a5197743-a5df-4c41-ba55-7dda189951f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:01.862946 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.862921    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654"]
Jul 23 03:59:01.865439 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:59:01.865415    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8de06117_90a8_4b8f_9966_b6e6ccfe164d.slice/crio-ff64b9ebd8751582a6ac4c053dced284f9c654ae3606b3979195747761fd18a0 WatchSource:0}: Error finding container ff64b9ebd8751582a6ac4c053dced284f9c654ae3606b3979195747761fd18a0: Status 404 returned error can't find the container with id ff64b9ebd8751582a6ac4c053dced284f9c654ae3606b3979195747761fd18a0
Jul 23 03:59:01.890385 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:01.890364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"
Jul 23 03:59:02.018972 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:02.018949    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt"]
Jul 23 03:59:02.020786 ip-10-0-138-244 kubenswrapper[2582]: W0723 03:59:02.020762    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5197743_a5df_4c41_ba55_7dda189951f2.slice/crio-424444f6249ebc30b17d15090d8c78212a3bd4fce4e91673f7a98f619d55110e WatchSource:0}: Error finding container 424444f6249ebc30b17d15090d8c78212a3bd4fce4e91673f7a98f619d55110e: Status 404 returned error can't find the container with id 424444f6249ebc30b17d15090d8c78212a3bd4fce4e91673f7a98f619d55110e
Jul 23 03:59:02.082360 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:02.082324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654" event={"ID":"8de06117-90a8-4b8f-9966-b6e6ccfe164d","Type":"ContainerStarted","Data":"ff64b9ebd8751582a6ac4c053dced284f9c654ae3606b3979195747761fd18a0"}
Jul 23 03:59:02.083314 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:02.083294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt" event={"ID":"a5197743-a5df-4c41-ba55-7dda189951f2","Type":"ContainerStarted","Data":"424444f6249ebc30b17d15090d8c78212a3bd4fce4e91673f7a98f619d55110e"}
Jul 23 03:59:06.103163 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:06.103126    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt" event={"ID":"a5197743-a5df-4c41-ba55-7dda189951f2","Type":"ContainerStarted","Data":"7dc6e7fa706d6ed0ab05e2bdd33eeeed859ea101c82209a1cd45b29337bb5d47"}
Jul 23 03:59:06.103594 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:06.103174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt" event={"ID":"a5197743-a5df-4c41-ba55-7dda189951f2","Type":"ContainerStarted","Data":"ac32c3dbb624a85217d0498a571f82d3c7df3def104149f9815bae50804f8748"}
Jul 23 03:59:06.103594 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:06.103192    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt" event={"ID":"a5197743-a5df-4c41-ba55-7dda189951f2","Type":"ContainerStarted","Data":"de356c57fe986a5354d59705fcd7f0de2d7f699ea70a0576ab7d11c27316d6a1"}
Jul 23 03:59:06.129309 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:06.129262    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c55996678-gbrrt" podStartSLOduration=1.6342602290000001 podStartE2EDuration="5.129247808s" totalImagesPullingTime="3.494987579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:59:06.128681766 +0000 UTC m=+228.589825440" watchObservedRunningTime="2026-07-23 03:59:06.129247808 +0000 UTC m=+228.590391479"
Jul 23 03:59:07.107874 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:07.107837    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654" event={"ID":"8de06117-90a8-4b8f-9966-b6e6ccfe164d","Type":"ContainerStarted","Data":"c17c7829226d10a1246dc76f703295ba0f44a2d8254611f9b050cb584357db2b"}
Jul 23 03:59:07.129338 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:07.129295    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d6547794-cd654" podStartSLOduration=1.651718319 podStartE2EDuration="6.129279995s" totalImagesPullingTime="4.477561676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 03:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 03:59:07.128874733 +0000 UTC m=+229.590018404" watchObservedRunningTime="2026-07-23 03:59:07.129279995 +0000 UTC m=+229.590423666"
Jul 23 03:59:17.491887 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:17.491846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:17.795893 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:17.795795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:21.429201 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:21.429160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:21.529398 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:21.529365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:22.895904 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:22.895868    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:23.069593 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:23.069560    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:23.287552 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:23.287513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:25.494204 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:25.494173    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:25.897256 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:25.897175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:27.394723 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:27.394689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:27.498106 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:27.498074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:33.218121 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:33.218090    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:33.749895 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:33.749861    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:33.850261 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:33.850226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:37.401528 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:37.401489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:37.509742 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:37.509697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:43.287027 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:43.286928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:43.386895 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:43.386869    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 03:59:45.379776 ip-10-0-138-244 kubenswrapper[2582]: I0723 03:59:45.379744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:00.584673 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:00.584635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:00.684028 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:00.683994    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:01.218191 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.218156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:01.609961 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.609926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:01.628211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.628183    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"]
Jul 23 04:00:01.630987 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.630972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.656757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.656722    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"]
Jul 23 04:00:01.670205 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.670180    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 04:00:01.670358 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.670218    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 04:00:01.670423 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.670379    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 04:00:01.671186 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.671172    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 04:00:01.672042 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.672029    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2clxt\""
Jul 23 04:00:01.755713 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.755679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktzhl\" (UniqueName: \"kubernetes.io/projected/64196160-52a7-4314-80d1-bcd328e53c90-kube-api-access-ktzhl\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.755911 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.755719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.755911 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.755783    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.755911 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.755872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.755911 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.755902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856403 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856508    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktzhl\" (UniqueName: \"kubernetes.io/projected/64196160-52a7-4314-80d1-bcd328e53c90-kube-api-access-ktzhl\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.856978 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.856954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-manifests\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.858826 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.858773    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64196160-52a7-4314-80d1-bcd328e53c90-tmp\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.858956 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.858937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.859090 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.859073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64196160-52a7-4314-80d1-bcd328e53c90-webhook-cert\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.874607 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.874539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktzhl\" (UniqueName: \"kubernetes.io/projected/64196160-52a7-4314-80d1-bcd328e53c90-kube-api-access-ktzhl\") pod \"opendatahub-operator-controller-manager-5775946f86-d2wsp\" (UID: \"64196160-52a7-4314-80d1-bcd328e53c90\") " pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.940703 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.940667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:01.960098 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:01.960064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:02.048169 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.048131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:02.090528 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.090477    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"]
Jul 23 04:00:02.092700 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:02.092674    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64196160_52a7_4314_80d1_bcd328e53c90.slice/crio-943a2adfe8e6340dfcda26b011f0317de5cc3dff43d34dd3a5738125d8532f99 WatchSource:0}: Error finding container 943a2adfe8e6340dfcda26b011f0317de5cc3dff43d34dd3a5738125d8532f99: Status 404 returned error can't find the container with id 943a2adfe8e6340dfcda26b011f0317de5cc3dff43d34dd3a5738125d8532f99
Jul 23 04:00:02.148868 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.148775    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:02.296174 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.296139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp" event={"ID":"64196160-52a7-4314-80d1-bcd328e53c90","Type":"ContainerStarted","Data":"943a2adfe8e6340dfcda26b011f0317de5cc3dff43d34dd3a5738125d8532f99"}
Jul 23 04:00:02.362923 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.362891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:02.462313 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.462230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:02.566160 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:02.566130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:06.316338 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:06.316302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp" event={"ID":"64196160-52a7-4314-80d1-bcd328e53c90","Type":"ContainerStarted","Data":"0769070302d16ccdbdcaa7cd1d665df3399db57a229fae8738ab36952c8adabb"}
Jul 23 04:00:07.324123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:07.324092    2582 generic.go:356] "Generic (PLEG): container finished" podID="64196160-52a7-4314-80d1-bcd328e53c90" containerID="0769070302d16ccdbdcaa7cd1d665df3399db57a229fae8738ab36952c8adabb" exitCode=0
Jul 23 04:00:07.324500 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:07.324141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp" event={"ID":"64196160-52a7-4314-80d1-bcd328e53c90","Type":"ContainerDied","Data":"0769070302d16ccdbdcaa7cd1d665df3399db57a229fae8738ab36952c8adabb"}
Jul 23 04:00:08.329895 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:08.329862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp" event={"ID":"64196160-52a7-4314-80d1-bcd328e53c90","Type":"ContainerStarted","Data":"8f069e42626a060eec1bdfc9c1d2c09ca38621de6f939c060efb36c215f638a4"}
Jul 23 04:00:08.330264 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:08.329978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:08.354542 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:08.354494    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp" podStartSLOduration=2.734885367 podStartE2EDuration="7.354476536s" totalImagesPullingTime="3.613459991s" totalInitContainerRuntime="1.006131178s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:08.353298243 +0000 UTC m=+290.814441915" watchObservedRunningTime="2026-07-23 04:00:08.354476536 +0000 UTC m=+290.815620248"
Jul 23 04:00:17.967860 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:17.967832    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 04:00:19.337023 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:19.336995    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5775946f86-d2wsp"
Jul 23 04:00:21.407160 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:21.407126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:21.542844 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:21.542795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:21.941506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:21.941466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:22.060025 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.059971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:22.235752 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.235721    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"]
Jul 23 04:00:22.238720 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.238704    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.247239 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.247219    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 04:00:22.249467 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.249453    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 04:00:22.249531 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.249494    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 04:00:22.251596 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.251580    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 04:00:22.252165 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.252152    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fjd7q\""
Jul 23 04:00:22.284677 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.284647    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"]
Jul 23 04:00:22.412875 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.412841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f06d7ac-1278-477c-888e-4242ef8756f2-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.412875 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.412877    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr426\" (UniqueName: \"kubernetes.io/projected/4f06d7ac-1278-477c-888e-4242ef8756f2-kube-api-access-lr426\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.413274 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.412907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f06d7ac-1278-477c-888e-4242ef8756f2-tmp\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.452917 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.452884    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q7clh"]
Jul 23 04:00:22.455894 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.455872    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.459900 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.459865    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 23 04:00:22.460304 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.460287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2th9s\""
Jul 23 04:00:22.474341 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.474315    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q7clh"]
Jul 23 04:00:22.503513 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.503440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:22.513597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.513575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f06d7ac-1278-477c-888e-4242ef8756f2-tmp\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.513730 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.513642    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f06d7ac-1278-477c-888e-4242ef8756f2-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.513796 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.513750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr426\" (UniqueName: \"kubernetes.io/projected/4f06d7ac-1278-477c-888e-4242ef8756f2-kube-api-access-lr426\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.515893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.515868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f06d7ac-1278-477c-888e-4242ef8756f2-tmp\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.515996 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.515981    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f06d7ac-1278-477c-888e-4242ef8756f2-tls-certs\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.528227 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.528208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr426\" (UniqueName: \"kubernetes.io/projected/4f06d7ac-1278-477c-888e-4242ef8756f2-kube-api-access-lr426\") pod \"kube-auth-proxy-c76cc57cd-qjb69\" (UID: \"4f06d7ac-1278-477c-888e-4242ef8756f2\") " pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.547964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.547945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"
Jul 23 04:00:22.614660 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.614625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdcf9\" (UniqueName: \"kubernetes.io/projected/281d5502-941f-4894-a8c9-139c52c30775-kube-api-access-kdcf9\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.614827 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.614712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.671272 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.671227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:22.701278 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.701253    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69"]
Jul 23 04:00:22.704099 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:22.704075    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f06d7ac_1278_477c_888e_4242ef8756f2.slice/crio-e7c74b9aa3a18f34383f6a65bf9a5ec4629ecafa8bf48b68ad3d081a8f3b5dbc WatchSource:0}: Error finding container e7c74b9aa3a18f34383f6a65bf9a5ec4629ecafa8bf48b68ad3d081a8f3b5dbc: Status 404 returned error can't find the container with id e7c74b9aa3a18f34383f6a65bf9a5ec4629ecafa8bf48b68ad3d081a8f3b5dbc
Jul 23 04:00:22.705779 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.705762    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:00:22.715637 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.715614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdcf9\" (UniqueName: \"kubernetes.io/projected/281d5502-941f-4894-a8c9-139c52c30775-kube-api-access-kdcf9\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.715734 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.715721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.715863 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:00:22.715848    2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 23 04:00:22.715935 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:00:22.715910    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert podName:281d5502-941f-4894-a8c9-139c52c30775 nodeName:}" failed. No retries permitted until 2026-07-23 04:00:23.215888937 +0000 UTC m=+305.677032598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert") pod "odh-model-controller-7998c67dc4-q7clh" (UID: "281d5502-941f-4894-a8c9-139c52c30775") : secret "odh-model-controller-webhook-cert" not found
Jul 23 04:00:22.739112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.739089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdcf9\" (UniqueName: \"kubernetes.io/projected/281d5502-941f-4894-a8c9-139c52c30775-kube-api-access-kdcf9\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:22.851582 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.851488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:22.956075 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:22.956043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:23.222497 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.222396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:23.224849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.224827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/281d5502-941f-4894-a8c9-139c52c30775-cert\") pod \"odh-model-controller-7998c67dc4-q7clh\" (UID: \"281d5502-941f-4894-a8c9-139c52c30775\") " pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:23.365730 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.365686    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:23.388035 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.388005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69" event={"ID":"4f06d7ac-1278-477c-888e-4242ef8756f2","Type":"ContainerStarted","Data":"e7c74b9aa3a18f34383f6a65bf9a5ec4629ecafa8bf48b68ad3d081a8f3b5dbc"}
Jul 23 04:00:23.498197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.498119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:23.537009 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:23.536976    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281d5502_941f_4894_a8c9_139c52c30775.slice/crio-14d2578834b85d2852054fef617b82e0a024c839654b31d6dbf6c838fe5b2190 WatchSource:0}: Error finding container 14d2578834b85d2852054fef617b82e0a024c839654b31d6dbf6c838fe5b2190: Status 404 returned error can't find the container with id 14d2578834b85d2852054fef617b82e0a024c839654b31d6dbf6c838fe5b2190
Jul 23 04:00:23.539892 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.539869    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q7clh"]
Jul 23 04:00:23.541573 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.541553    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5xv"]
Jul 23 04:00:23.544685 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.544667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.547951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.547932    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 04:00:23.553349 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.553330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5xv"]
Jul 23 04:00:23.727074 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.727042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n6sh\" (UniqueName: \"kubernetes.io/projected/b2b76864-a24b-4d0c-90e3-6a55d353b877-kube-api-access-8n6sh\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.727218 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.727137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.762135 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.762037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:23.762477 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.762453    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:23.766093 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.766075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.768517 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.768488    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 04:00:23.768653 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.768634    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 04:00:23.768751 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.768735    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 04:00:23.768832 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.768750    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dh6n8\""
Jul 23 04:00:23.768832 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.768788    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 04:00:23.778379 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.778357    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:23.827912 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.827881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n6sh\" (UniqueName: \"kubernetes.io/projected/b2b76864-a24b-4d0c-90e3-6a55d353b877-kube-api-access-8n6sh\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.828066 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.827963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.828141 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:00:23.828110    2582 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 04:00:23.828198 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:00:23.828167    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs podName:b2b76864-a24b-4d0c-90e3-6a55d353b877 nodeName:}" failed. No retries permitted until 2026-07-23 04:00:24.32814714 +0000 UTC m=+306.789290789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs") pod "model-serving-api-54d89bd5fc-cm5xv" (UID: "b2b76864-a24b-4d0c-90e3-6a55d353b877") : secret "model-serving-api-tls" not found
Jul 23 04:00:23.839777 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.839751    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n6sh\" (UniqueName: \"kubernetes.io/projected/b2b76864-a24b-4d0c-90e3-6a55d353b877-kube-api-access-8n6sh\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:23.928412 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928572 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928572 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4kq8\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928572 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928739 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928739 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:23.928739 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:23.928719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029346 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029346 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4kq8\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029459    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.029533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.029522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.030243 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.030215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.031878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.031852    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.031987 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.031936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.032056 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.032043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.032194 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.032177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.037104 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.037082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.037526 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.037507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4kq8\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8\") pod \"istiod-openshift-gateway-7574459656-zhtw2\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.077674 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.077649    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:24.235104 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.234980    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:24.237904 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:24.237876    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7370747_21f5_453b_90e8_e169fa320e54.slice/crio-456be4ff59c4ab772adac046e24c10ffc9af26762320b9b30b340936ca3064c6 WatchSource:0}: Error finding container 456be4ff59c4ab772adac046e24c10ffc9af26762320b9b30b340936ca3064c6: Status 404 returned error can't find the container with id 456be4ff59c4ab772adac046e24c10ffc9af26762320b9b30b340936ca3064c6
Jul 23 04:00:24.332193 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.331758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:24.336008 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.335906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2b76864-a24b-4d0c-90e3-6a55d353b877-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5xv\" (UID: \"b2b76864-a24b-4d0c-90e3-6a55d353b877\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:24.397065 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.397028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" event={"ID":"e7370747-21f5-453b-90e8-e169fa320e54","Type":"ContainerStarted","Data":"456be4ff59c4ab772adac046e24c10ffc9af26762320b9b30b340936ca3064c6"}
Jul 23 04:00:24.398371 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.398346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh" event={"ID":"281d5502-941f-4894-a8c9-139c52c30775","Type":"ContainerStarted","Data":"14d2578834b85d2852054fef617b82e0a024c839654b31d6dbf6c838fe5b2190"}
Jul 23 04:00:24.457792 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.457753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:24.652827 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.652772    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5xv"]
Jul 23 04:00:24.655485 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:24.655121    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b76864_a24b_4d0c_90e3_6a55d353b877.slice/crio-c181c10ffbdd1791ca6980a371162bb6b46fc7aa501837090c6ae97596146180 WatchSource:0}: Error finding container c181c10ffbdd1791ca6980a371162bb6b46fc7aa501837090c6ae97596146180: Status 404 returned error can't find the container with id c181c10ffbdd1791ca6980a371162bb6b46fc7aa501837090c6ae97596146180
Jul 23 04:00:24.663673 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:24.663649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:25.404004 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:25.403958    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv" event={"ID":"b2b76864-a24b-4d0c-90e3-6a55d353b877","Type":"ContainerStarted","Data":"c181c10ffbdd1791ca6980a371162bb6b46fc7aa501837090c6ae97596146180"}
Jul 23 04:00:26.217953 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:26.217916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:28.397311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.397277    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:00:28.397674 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.397359    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:00:28.421584 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.421558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh" event={"ID":"281d5502-941f-4894-a8c9-139c52c30775","Type":"ContainerStarted","Data":"552b85af4b837f9854efc1c557b7f611d92fbca05a473ad3478423617ac24236"}
Jul 23 04:00:28.421696 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.421645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:28.423302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.423280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv" event={"ID":"b2b76864-a24b-4d0c-90e3-6a55d353b877","Type":"ContainerStarted","Data":"ce204073ca26b3558d786856d80ab59f6e853eaacab1160f71625f4c3d8ec5b9"}
Jul 23 04:00:28.423392 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.423332    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:00:28.424898 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.424875    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69" event={"ID":"4f06d7ac-1278-477c-888e-4242ef8756f2","Type":"ContainerStarted","Data":"894145fd5a014aeb6ce20568ef68465295d94ceccaf7d16806a5bb2a771f4797"}
Jul 23 04:00:28.444053 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.444005    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh" podStartSLOduration=1.6488992260000002 podStartE2EDuration="6.443991913s" totalImagesPullingTime="4.795092687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:28.443558243 +0000 UTC m=+310.904701937" watchObservedRunningTime="2026-07-23 04:00:28.443991913 +0000 UTC m=+310.905135584"
Jul 23 04:00:28.462818 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.462754    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv" podStartSLOduration=1.779187868 podStartE2EDuration="5.462735169s" totalImagesPullingTime="3.683547301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:28.461673965 +0000 UTC m=+310.922817663" watchObservedRunningTime="2026-07-23 04:00:28.462735169 +0000 UTC m=+310.923878848"
Jul 23 04:00:28.482546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.482495    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c76cc57cd-qjb69" podStartSLOduration=2.297555515 podStartE2EDuration="6.482478166s" totalImagesPullingTime="4.184922651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:28.481246097 +0000 UTC m=+310.942389784" watchObservedRunningTime="2026-07-23 04:00:28.482478166 +0000 UTC m=+310.943621837"
Jul 23 04:00:28.954859 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:28.954822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:29.093742 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:29.093698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:29.435344 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:29.435300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" event={"ID":"e7370747-21f5-453b-90e8-e169fa320e54","Type":"ContainerStarted","Data":"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"}
Jul 23 04:00:29.460110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:29.459927    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" podStartSLOduration=2.302908951 podStartE2EDuration="6.459914369s" totalImagesPullingTime="4.157005418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:29.4597601 +0000 UTC m=+311.920903774" watchObservedRunningTime="2026-07-23 04:00:29.459914369 +0000 UTC m=+311.921058051"
Jul 23 04:00:29.882493 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:29.882440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:29.976185 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:29.976150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:30.119323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.119283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:30.354257 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.354224    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"]
Jul 23 04:00:30.357592 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.357572    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.374951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.374930    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"]
Jul 23 04:00:30.376681 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376784 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376784 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kxq\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-kube-api-access-m6kxq\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376920 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376920 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376998 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.376998 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.376968    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.440017 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.439995    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:30.441328 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.441307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:30.478107 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kxq\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-kube-api-access-m6kxq\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.478233 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.478233 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.478381 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.478436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.478614 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.478594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.479265 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.479209    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.479369 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.479330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.480997 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.480958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.481178 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.481153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.481391 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.481370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.481869 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.481850    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.488245 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.488225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kxq\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-kube-api-access-m6kxq\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.489725 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.489690    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e5f9c-fb5f-4d56-b778-fcaa96ff2b27-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-469qn\" (UID: \"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.666846 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.666762    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:30.807490 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:00:30.807457    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444e5f9c_fb5f_4d56_b778_fcaa96ff2b27.slice/crio-5bcb9cc302b61f5f46738591818ae1e4765ecc9502b8a9d09a577738e0543f9e WatchSource:0}: Error finding container 5bcb9cc302b61f5f46738591818ae1e4765ecc9502b8a9d09a577738e0543f9e: Status 404 returned error can't find the container with id 5bcb9cc302b61f5f46738591818ae1e4765ecc9502b8a9d09a577738e0543f9e
Jul 23 04:00:30.811133 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.811094    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:00:30.811235 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.811173    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:00:30.811749 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:30.811727    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"]
Jul 23 04:00:31.445937 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.445902    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn" event={"ID":"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27","Type":"ContainerStarted","Data":"c2dbe196176e3318ff312b8ad1b4c356b7bd5ffe93fdb97aad64367c1ef3c203"}
Jul 23 04:00:31.446320 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.445959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn" event={"ID":"444e5f9c-fb5f-4d56-b778-fcaa96ff2b27","Type":"ContainerStarted","Data":"5bcb9cc302b61f5f46738591818ae1e4765ecc9502b8a9d09a577738e0543f9e"}
Jul 23 04:00:31.446320 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.446157    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:31.447655 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.447627    2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-469qn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 04:00:31.447757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.447684    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn" podUID="444e5f9c-fb5f-4d56-b778-fcaa96ff2b27" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 04:00:31.499743 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:31.499690    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn" podStartSLOduration=1.49966901 podStartE2EDuration="1.49966901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:00:31.496038848 +0000 UTC m=+313.957182520" watchObservedRunningTime="2026-07-23 04:00:31.49966901 +0000 UTC m=+313.960812708"
Jul 23 04:00:32.451079 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:32.451053    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-469qn"
Jul 23 04:00:32.537471 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:32.537443    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:33.456096 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.456046    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" podUID="e7370747-21f5-453b-90e8-e169fa320e54" containerName="discovery" containerID="cri-o://39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c" gracePeriod=30
Jul 23 04:00:33.462058 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.462027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:33.613236 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.613115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:33.705161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.705133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:33.711473 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.711425    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:33.807972 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.807946    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808025    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808085    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4kq8\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808109    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808310 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808142    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808310 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808194    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs\") pod \"e7370747-21f5-453b-90e8-e169fa320e54\" (UID: \"e7370747-21f5-453b-90e8-e169fa320e54\") "
Jul 23 04:00:33.808879 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.808795    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:00:33.810588 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810555    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs" (OuterVolumeSpecName: "local-certs") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:00:33.810731 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810687    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts" (OuterVolumeSpecName: "cacerts") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:00:33.810731 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810706    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:00:33.810887 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810763    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:00:33.810887 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810822    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8" (OuterVolumeSpecName: "kube-api-access-h4kq8") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "kube-api-access-h4kq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:00:33.810962 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.810931    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7370747-21f5-453b-90e8-e169fa320e54" (UID: "e7370747-21f5-453b-90e8-e169fa320e54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:00:33.832451 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.832422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:33.909250 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909228    2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-kubeconfig\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909250 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909248    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-dns-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909260    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7370747-21f5-453b-90e8-e169fa320e54-istio-csr-ca-configmap\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909270    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4kq8\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-kube-api-access-h4kq8\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909282    2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7370747-21f5-453b-90e8-e169fa320e54-cacerts\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909294    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7370747-21f5-453b-90e8-e169fa320e54-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.909377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.909303    2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7370747-21f5-453b-90e8-e169fa320e54-local-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:00:33.966588 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:33.966532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:34.091705 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.091678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:00:34.461408 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.461381    2582 generic.go:356] "Generic (PLEG): container finished" podID="e7370747-21f5-453b-90e8-e169fa320e54" containerID="39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c" exitCode=0
Jul 23 04:00:34.461849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.461421    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"
Jul 23 04:00:34.461849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.461426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" event={"ID":"e7370747-21f5-453b-90e8-e169fa320e54","Type":"ContainerDied","Data":"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"}
Jul 23 04:00:34.461849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.461471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2" event={"ID":"e7370747-21f5-453b-90e8-e169fa320e54","Type":"ContainerDied","Data":"456be4ff59c4ab772adac046e24c10ffc9af26762320b9b30b340936ca3064c6"}
Jul 23 04:00:34.461849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.461494    2582 scope.go:117] "RemoveContainer" containerID="39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"
Jul 23 04:00:34.470061 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.470048    2582 scope.go:117] "RemoveContainer" containerID="39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"
Jul 23 04:00:34.470318 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:00:34.470294    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c\": container with ID starting with 39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c not found: ID does not exist" containerID="39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"
Jul 23 04:00:34.470385 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.470326    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c"} err="failed to get container status \"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c\": rpc error: code = NotFound desc = could not find container \"39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c\": container with ID starting with 39ac77ec597dbeeb998fe3a8ece073aa360da2a653a1149757a231c9284bce5c not found: ID does not exist"
Jul 23 04:00:34.492854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.492828    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:34.496629 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:34.496606    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zhtw2"]
Jul 23 04:00:36.051547 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:36.051516    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7370747-21f5-453b-90e8-e169fa320e54" path="/var/lib/kubelet/pods/e7370747-21f5-453b-90e8-e169fa320e54/volumes"
Jul 23 04:00:39.438009 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:39.437980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-q7clh"
Jul 23 04:00:39.439888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:00:39.439868    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cm5xv"
Jul 23 04:01:13.665858 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:13.665822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:22.860620 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:22.860583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:22.960326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:22.960295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:23.005599 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.005569    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:23.005907 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.005894    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7370747-21f5-453b-90e8-e169fa320e54" containerName="discovery"
Jul 23 04:01:23.005960 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.005908    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7370747-21f5-453b-90e8-e169fa320e54" containerName="discovery"
Jul 23 04:01:23.005995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.005961    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7370747-21f5-453b-90e8-e169fa320e54" containerName="discovery"
Jul 23 04:01:23.009834 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.009794    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:23.012207 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.012188    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 04:01:23.014606 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.014585    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b47wl\""
Jul 23 04:01:23.014606 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.014601    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 04:01:23.019544 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.019523    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:23.061398 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.061374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:23.169734 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.169675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgnl8\" (UniqueName: \"kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8\") pod \"kuadrant-operator-catalog-58wf2\" (UID: \"b0f57a44-00bf-40a1-8644-bd2f78b9106f\") " pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:23.270747 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.270722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgnl8\" (UniqueName: \"kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8\") pod \"kuadrant-operator-catalog-58wf2\" (UID: \"b0f57a44-00bf-40a1-8644-bd2f78b9106f\") " pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:23.279005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.278980    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgnl8\" (UniqueName: \"kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8\") pod \"kuadrant-operator-catalog-58wf2\" (UID: \"b0f57a44-00bf-40a1-8644-bd2f78b9106f\") " pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:23.320795 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.320771    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:23.364454 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.364427    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:23.460003 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.459980    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:23.462240 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:01:23.462213    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f57a44_00bf_40a1_8644_bd2f78b9106f.slice/crio-be78c6dfd3a4e7bd90759df0ea455d576bdb18db5b16483b1bac96a49f2ee817 WatchSource:0}: Error finding container be78c6dfd3a4e7bd90759df0ea455d576bdb18db5b16483b1bac96a49f2ee817: Status 404 returned error can't find the container with id be78c6dfd3a4e7bd90759df0ea455d576bdb18db5b16483b1bac96a49f2ee817
Jul 23 04:01:23.576469 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.576440    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lkx"]
Jul 23 04:01:23.580652 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.580637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:23.587577 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.587551    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lkx"]
Jul 23 04:01:23.659090 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.659059    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" event={"ID":"b0f57a44-00bf-40a1-8644-bd2f78b9106f","Type":"ContainerStarted","Data":"be78c6dfd3a4e7bd90759df0ea455d576bdb18db5b16483b1bac96a49f2ee817"}
Jul 23 04:01:23.674393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.674367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44s6k\" (UniqueName: \"kubernetes.io/projected/b3e66f3c-1a25-4d86-bdf8-5452063d92b5-kube-api-access-44s6k\") pod \"kuadrant-operator-catalog-c8lkx\" (UID: \"b3e66f3c-1a25-4d86-bdf8-5452063d92b5\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:23.774957 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.774931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44s6k\" (UniqueName: \"kubernetes.io/projected/b3e66f3c-1a25-4d86-bdf8-5452063d92b5-kube-api-access-44s6k\") pod \"kuadrant-operator-catalog-c8lkx\" (UID: \"b3e66f3c-1a25-4d86-bdf8-5452063d92b5\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:23.783218 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.783194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44s6k\" (UniqueName: \"kubernetes.io/projected/b3e66f3c-1a25-4d86-bdf8-5452063d92b5-kube-api-access-44s6k\") pod \"kuadrant-operator-catalog-c8lkx\" (UID: \"b3e66f3c-1a25-4d86-bdf8-5452063d92b5\") " pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:23.797877 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.797829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:23.890899 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:23.890873    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:24.009074 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:24.009029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:24.018516 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:24.018435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8lkx"]
Jul 23 04:01:24.021472 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:01:24.021441    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e66f3c_1a25_4d86_bdf8_5452063d92b5.slice/crio-b86ff3f37ae9d3a045f5617f2bbe8b8715a823d401cea76ae3a5107bbcd80ee2 WatchSource:0}: Error finding container b86ff3f37ae9d3a045f5617f2bbe8b8715a823d401cea76ae3a5107bbcd80ee2: Status 404 returned error can't find the container with id b86ff3f37ae9d3a045f5617f2bbe8b8715a823d401cea76ae3a5107bbcd80ee2
Jul 23 04:01:24.666394 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:24.666354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx" event={"ID":"b3e66f3c-1a25-4d86-bdf8-5452063d92b5","Type":"ContainerStarted","Data":"b86ff3f37ae9d3a045f5617f2bbe8b8715a823d401cea76ae3a5107bbcd80ee2"}
Jul 23 04:01:26.677907 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.677863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx" event={"ID":"b3e66f3c-1a25-4d86-bdf8-5452063d92b5","Type":"ContainerStarted","Data":"36afce273d77553cd04d037628ee1a7a96cf4890ca3ff6e852e744577c4433eb"}
Jul 23 04:01:26.679307 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.679278    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" event={"ID":"b0f57a44-00bf-40a1-8644-bd2f78b9106f","Type":"ContainerStarted","Data":"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"}
Jul 23 04:01:26.679426 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.679389    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" podUID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" containerName="registry-server" containerID="cri-o://a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a" gracePeriod=2
Jul 23 04:01:26.695993 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.695951    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx" podStartSLOduration=2.089568035 podStartE2EDuration="3.695937144s" totalImagesPullingTime="1.606369109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:01:26.69439215 +0000 UTC m=+369.155535823" watchObservedRunningTime="2026-07-23 04:01:26.695937144 +0000 UTC m=+369.157080815"
Jul 23 04:01:26.710026 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.709990    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" podStartSLOduration=2.544630115 podStartE2EDuration="4.709976641s" totalImagesPullingTime="2.165346526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:01:26.70967294 +0000 UTC m=+369.170816610" watchObservedRunningTime="2026-07-23 04:01:26.709976641 +0000 UTC m=+369.171120312"
Jul 23 04:01:26.919706 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:26.919682    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:27.101098 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.101071    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgnl8\" (UniqueName: \"kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8\") pod \"b0f57a44-00bf-40a1-8644-bd2f78b9106f\" (UID: \"b0f57a44-00bf-40a1-8644-bd2f78b9106f\") "
Jul 23 04:01:27.103132 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.103102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8" (OuterVolumeSpecName: "kube-api-access-wgnl8") pod "b0f57a44-00bf-40a1-8644-bd2f78b9106f" (UID: "b0f57a44-00bf-40a1-8644-bd2f78b9106f"). InnerVolumeSpecName "kube-api-access-wgnl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:01:27.202265 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.202224    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgnl8\" (UniqueName: \"kubernetes.io/projected/b0f57a44-00bf-40a1-8644-bd2f78b9106f-kube-api-access-wgnl8\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:01:27.684037 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.684000    2582 generic.go:356] "Generic (PLEG): container finished" podID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" containerID="a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a" exitCode=0
Jul 23 04:01:27.684464 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.684044    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-58wf2"
Jul 23 04:01:27.684464 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.684091    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" event={"ID":"b0f57a44-00bf-40a1-8644-bd2f78b9106f","Type":"ContainerDied","Data":"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"}
Jul 23 04:01:27.684464 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.684135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-58wf2" event={"ID":"b0f57a44-00bf-40a1-8644-bd2f78b9106f","Type":"ContainerDied","Data":"be78c6dfd3a4e7bd90759df0ea455d576bdb18db5b16483b1bac96a49f2ee817"}
Jul 23 04:01:27.684464 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.684155    2582 scope.go:117] "RemoveContainer" containerID="a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"
Jul 23 04:01:27.692938 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.692919    2582 scope.go:117] "RemoveContainer" containerID="a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"
Jul 23 04:01:27.693192 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:01:27.693175    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a\": container with ID starting with a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a not found: ID does not exist" containerID="a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"
Jul 23 04:01:27.693253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.693198    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a"} err="failed to get container status \"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a\": rpc error: code = NotFound desc = could not find container \"a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a\": container with ID starting with a562340d9b2f77644bc911f3f689880973e38b3debd50e2e8297e4c2dd71275a not found: ID does not exist"
Jul 23 04:01:27.704721 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.704699    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:27.708670 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:27.708651    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-58wf2"]
Jul 23 04:01:28.052266 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:28.052241    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" path="/var/lib/kubelet/pods/b0f57a44-00bf-40a1-8644-bd2f78b9106f/volumes"
Jul 23 04:01:33.891540 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:33.891504    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:33.891963 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:33.891577    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:33.913627 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:33.913600    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:34.735759 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:34.735736    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-c8lkx"
Jul 23 04:01:51.949795 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:51.949745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:54.383243 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.383211    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"]
Jul 23 04:01:54.383597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.383504    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" containerName="registry-server"
Jul 23 04:01:54.383597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.383515    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" containerName="registry-server"
Jul 23 04:01:54.383597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.383562    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f57a44-00bf-40a1-8644-bd2f78b9106f" containerName="registry-server"
Jul 23 04:01:54.386635 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.386619    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:54.389974 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.389956    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4thbv\""
Jul 23 04:01:54.390952 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.390932    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 04:01:54.400956 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.400934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgp6t\" (UniqueName: \"kubernetes.io/projected/02110674-69f4-475e-afa6-323a61f4d622-kube-api-access-rgp6t\") pod \"dns-operator-controller-manager-864dd78c4c-z6q5q\" (UID: \"02110674-69f4-475e-afa6-323a61f4d622\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:54.402186 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.402167    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"]
Jul 23 04:01:54.502118 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.502090    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgp6t\" (UniqueName: \"kubernetes.io/projected/02110674-69f4-475e-afa6-323a61f4d622-kube-api-access-rgp6t\") pod \"dns-operator-controller-manager-864dd78c4c-z6q5q\" (UID: \"02110674-69f4-475e-afa6-323a61f4d622\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:54.514016 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.513986    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgp6t\" (UniqueName: \"kubernetes.io/projected/02110674-69f4-475e-afa6-323a61f4d622-kube-api-access-rgp6t\") pod \"dns-operator-controller-manager-864dd78c4c-z6q5q\" (UID: \"02110674-69f4-475e-afa6-323a61f4d622\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:54.696179 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.696104    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:54.827127 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:54.827092    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"]
Jul 23 04:01:54.833313 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:01:54.833277    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02110674_69f4_475e_afa6_323a61f4d622.slice/crio-4c182b506ad44c146c2f6a6112700c0748725ab5c19e3fd0b595090b91a73e38 WatchSource:0}: Error finding container 4c182b506ad44c146c2f6a6112700c0748725ab5c19e3fd0b595090b91a73e38: Status 404 returned error can't find the container with id 4c182b506ad44c146c2f6a6112700c0748725ab5c19e3fd0b595090b91a73e38
Jul 23 04:01:55.804670 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:55.804631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q" event={"ID":"02110674-69f4-475e-afa6-323a61f4d622","Type":"ContainerStarted","Data":"4c182b506ad44c146c2f6a6112700c0748725ab5c19e3fd0b595090b91a73e38"}
Jul 23 04:01:55.953551 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:55.953506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:57.815244 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.815211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q" event={"ID":"02110674-69f4-475e-afa6-323a61f4d622","Type":"ContainerStarted","Data":"66977af39258b10762409bef5cef8ec1698f170e374ef92b95a6b4ba359b2593"}
Jul 23 04:01:57.815608 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.815268    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:01:57.844756 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.844711    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q" podStartSLOduration=1.924776037 podStartE2EDuration="3.844696719s" totalImagesPullingTime="1.919920682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:01:57.844010304 +0000 UTC m=+400.305153976" watchObservedRunningTime="2026-07-23 04:01:57.844696719 +0000 UTC m=+400.305840409"
Jul 23 04:01:57.904110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.904068    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"]
Jul 23 04:01:57.907456 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.907438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:01:57.909983 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.909958    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fsm4b\""
Jul 23 04:01:57.918391 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.918370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"]
Jul 23 04:01:57.932650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:57.932624    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptq9\" (UniqueName: \"kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9\") pod \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" (UID: \"892b896e-f397-4ad1-bafb-4b2eec73aaa5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:01:58.033559 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.033530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptq9\" (UniqueName: \"kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9\") pod \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" (UID: \"892b896e-f397-4ad1-bafb-4b2eec73aaa5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:01:58.043223 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.043190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptq9\" (UniqueName: \"kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9\") pod \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" (UID: \"892b896e-f397-4ad1-bafb-4b2eec73aaa5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:01:58.217441 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.217371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:01:58.348060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.348019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:58.354133 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.354111    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"]
Jul 23 04:01:58.355912 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:01:58.355890    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892b896e_f397_4ad1_bafb_4b2eec73aaa5.slice/crio-9f7b65daf6d4a2f7b28852ee09c5ba1adf376a986df296c5c947a7128d92248a WatchSource:0}: Error finding container 9f7b65daf6d4a2f7b28852ee09c5ba1adf376a986df296c5c947a7128d92248a: Status 404 returned error can't find the container with id 9f7b65daf6d4a2f7b28852ee09c5ba1adf376a986df296c5c947a7128d92248a
Jul 23 04:01:58.598568 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.598538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:01:58.820442 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.820408    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7" event={"ID":"892b896e-f397-4ad1-bafb-4b2eec73aaa5","Type":"ContainerStarted","Data":"9f7b65daf6d4a2f7b28852ee09c5ba1adf376a986df296c5c947a7128d92248a"}
Jul 23 04:01:58.961822 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:01:58.961727    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:00.830895 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:00.830852    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7" event={"ID":"892b896e-f397-4ad1-bafb-4b2eec73aaa5","Type":"ContainerStarted","Data":"d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643"}
Jul 23 04:02:00.831275 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:00.830988    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:00.849061 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:00.849020    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7" podStartSLOduration=2.166531714 podStartE2EDuration="3.849006742s" totalImagesPullingTime="1.682475028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:00.84810966 +0000 UTC m=+403.309253332" watchObservedRunningTime="2026-07-23 04:02:00.849006742 +0000 UTC m=+403.310150413"
Jul 23 04:02:02.345315 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:02.345279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:03.450367 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:03.450329    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:05.422854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:05.422793    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:06.672262 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.672229    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"]
Jul 23 04:02:06.675540 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.675525    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.678725 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.678707    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qrd4w\""
Jul 23 04:02:06.700694 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.700673    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"]
Jul 23 04:02:06.801103 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.801075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdb22\" (UniqueName: \"kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.801224 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.801116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.902397 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.902368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.902540 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.902430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdb22\" (UniqueName: \"kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.902849 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.902783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.910632 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.910614    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdb22\" (UniqueName: \"kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:06.985690 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:06.985662    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:07.119692 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:07.119662    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"]
Jul 23 04:02:07.122664 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:02:07.122634    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb84556_21ef_4b53_ad32_768fc14135cc.slice/crio-146df8d231b48c0e2ac184ca9d6702234417d7d5012c5347e2d0df1a0105ea88 WatchSource:0}: Error finding container 146df8d231b48c0e2ac184ca9d6702234417d7d5012c5347e2d0df1a0105ea88: Status 404 returned error can't find the container with id 146df8d231b48c0e2ac184ca9d6702234417d7d5012c5347e2d0df1a0105ea88
Jul 23 04:02:07.201219 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:07.201191    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:07.862176 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:07.862141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc" event={"ID":"efb84556-21ef-4b53-ad32-768fc14135cc","Type":"ContainerStarted","Data":"146df8d231b48c0e2ac184ca9d6702234417d7d5012c5347e2d0df1a0105ea88"}
Jul 23 04:02:08.635459 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:08.635426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:08.735590 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:08.735554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:08.822742 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:08.822714    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z6q5q"
Jul 23 04:02:09.774854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.774821    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f455bbfc8-9kwh7"]
Jul 23 04:02:09.778537 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.778515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.798770 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.798743    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f455bbfc8-9kwh7"]
Jul 23 04:02:09.825050 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-oauth-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825176 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-service-ca\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825176 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825146    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-oauth-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-trusted-ca-bundle\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bsc\" (UniqueName: \"kubernetes.io/projected/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-kube-api-access-b4bsc\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.825302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.825269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.925821 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-oauth-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.925975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925834    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-trusted-ca-bundle\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.925975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.925975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bsc\" (UniqueName: \"kubernetes.io/projected/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-kube-api-access-b4bsc\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.925975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.926182 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.925997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-oauth-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.926182 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.926020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-service-ca\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.926610 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.926579    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-oauth-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.927061 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.927011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-trusted-ca-bundle\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.927225 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.927200    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.927353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.927252    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-service-ca\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.929012 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.928991    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-serving-cert\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.929235 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.929211    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-console-oauth-config\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:09.932537 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.932502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:09.935256 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:09.935235    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bsc\" (UniqueName: \"kubernetes.io/projected/ba118e9b-ff2c-4513-aeb3-4ea987feb0d1-kube-api-access-b4bsc\") pod \"console-6f455bbfc8-9kwh7\" (UID: \"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1\") " pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:10.090646 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.090580    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:10.241270 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.241247    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f455bbfc8-9kwh7"]
Jul 23 04:02:10.243163 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:02:10.243139    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba118e9b_ff2c_4513_aeb3_4ea987feb0d1.slice/crio-309150782ee5127ca9dd7eeb8e708d6ab07bc39949235b5e2db9830a82e3f48f WatchSource:0}: Error finding container 309150782ee5127ca9dd7eeb8e708d6ab07bc39949235b5e2db9830a82e3f48f: Status 404 returned error can't find the container with id 309150782ee5127ca9dd7eeb8e708d6ab07bc39949235b5e2db9830a82e3f48f
Jul 23 04:02:10.398266 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.398185    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:10.543369 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.543338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:10.609946 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.609912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:10.880406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.880372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f455bbfc8-9kwh7" event={"ID":"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1","Type":"ContainerStarted","Data":"1c1396673addc84e43480e599936388c002b33b59351466ac21a4248fddb98b3"}
Jul 23 04:02:10.880862 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.880426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f455bbfc8-9kwh7" event={"ID":"ba118e9b-ff2c-4513-aeb3-4ea987feb0d1","Type":"ContainerStarted","Data":"309150782ee5127ca9dd7eeb8e708d6ab07bc39949235b5e2db9830a82e3f48f"}
Jul 23 04:02:10.908222 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:10.908176    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f455bbfc8-9kwh7" podStartSLOduration=1.908158815 podStartE2EDuration="1.908158815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:10.907560978 +0000 UTC m=+413.368704650" watchObservedRunningTime="2026-07-23 04:02:10.908158815 +0000 UTC m=+413.369302487"
Jul 23 04:02:11.837925 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:11.837898    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:11.889936 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:11.889902    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc" event={"ID":"efb84556-21ef-4b53-ad32-768fc14135cc","Type":"ContainerStarted","Data":"5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d"}
Jul 23 04:02:11.909410 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:11.909364    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc" podStartSLOduration=1.253042986 podStartE2EDuration="5.909349212s" totalImagesPullingTime="4.656306226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:11.907463888 +0000 UTC m=+414.368607561" watchObservedRunningTime="2026-07-23 04:02:11.909349212 +0000 UTC m=+414.370492884"
Jul 23 04:02:12.383899 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:12.383862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:12.480193 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:12.480155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:12.894192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:12.894161    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:14.417192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:14.417157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:15.037917 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:15.037879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:16.551935 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:16.551902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:16.656403 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:16.656374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:16.965475 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:16.965388    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:17.073647 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:17.073615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:20.091088 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.091060    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:20.091460 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.091300    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:20.095925 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.095906    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:20.799773 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.799742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:20.900171 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.900138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:20.935645 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.935619    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f455bbfc8-9kwh7"
Jul 23 04:02:20.994568 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:20.994537    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 04:02:21.110437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:21.110360    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:21.210923 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:21.210888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:21.276981 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:21.276953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:23.900716 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:23.900685    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.634105 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.634072    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"]
Jul 23 04:02:25.634527 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.634241    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" containerName="manager" containerID="cri-o://5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d" gracePeriod=2
Jul 23 04:02:25.644957 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.644932    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"]
Jul 23 04:02:25.647035 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.646993    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"efb84556-21ef-4b53-ad32-768fc14135cc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.663027 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.663001    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"]
Jul 23 04:02:25.663231 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.663208    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" containerName="manager" containerID="cri-o://d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643" gracePeriod=2
Jul 23 04:02:25.664777 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.664751    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:25.665095 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.665082    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" containerName="manager"
Jul 23 04:02:25.665149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.665098    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" containerName="manager"
Jul 23 04:02:25.665186 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.665176    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" containerName="manager"
Jul 23 04:02:25.668120 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.668102    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.686778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.686753    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"]
Jul 23 04:02:25.688942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.688920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:25.692923 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.692896    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.694951 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.694926    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.754019 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.753995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.754130 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.754050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7rpk\" (UniqueName: \"kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.855005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.854970    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rpk\" (UniqueName: \"kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.855151 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.855093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.855548 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.855523    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.864497 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.864472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7rpk\" (UniqueName: \"kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmpbv\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:25.903527 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.903508    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.906083 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.906059    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.906271 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.906256    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:25.908357 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.908336    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.910360 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.910340    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:25.955420 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.955395    2582 generic.go:356] "Generic (PLEG): container finished" podID="efb84556-21ef-4b53-ad32-768fc14135cc" containerID="5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d" exitCode=0
Jul 23 04:02:25.955518 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.955430    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.955518 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.955470    2582 scope.go:117] "RemoveContainer" containerID="5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d"
Jul 23 04:02:25.956773 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.956754    2582 generic.go:356] "Generic (PLEG): container finished" podID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" containerID="d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643" exitCode=0
Jul 23 04:02:25.956883 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.956799    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:25.957879 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.957858    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.959952 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.959921    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:25.961962 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.961940    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjvd7\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjvd7"
Jul 23 04:02:25.963796 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.963776    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ml9hc\" is forbidden: User \"system:node:ip-10-0-138-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-244.ec2.internal' and this object" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ml9hc"
Jul 23 04:02:25.964701 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.964682    2582 scope.go:117] "RemoveContainer" containerID="5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d"
Jul 23 04:02:25.965056 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.965033    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d\": container with ID starting with 5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d not found: ID does not exist" containerID="5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d"
Jul 23 04:02:25.965143 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.965064    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d"} err="failed to get container status \"5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d\": rpc error: code = NotFound desc = could not find container \"5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d\": container with ID starting with 5ba5632479adf2b81a961b1c63a2a47407001b601785789d2bd2e917a722878d not found: ID does not exist"
Jul 23 04:02:25.965143 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.965083    2582 scope.go:117] "RemoveContainer" containerID="d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643"
Jul 23 04:02:25.972060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.972042    2582 scope.go:117] "RemoveContainer" containerID="d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643"
Jul 23 04:02:25.972297 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:25.972280    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643\": container with ID starting with d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643 not found: ID does not exist" containerID="d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643"
Jul 23 04:02:25.972340 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:25.972304    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643"} err="failed to get container status \"d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643\": rpc error: code = NotFound desc = could not find container \"d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643\": container with ID starting with d1d3e9b923ddadcede4ccc094d62da4db25f90a36765d16048770801ea92e643 not found: ID does not exist"
Jul 23 04:02:26.056573 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.056546    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdb22\" (UniqueName: \"kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22\") pod \"efb84556-21ef-4b53-ad32-768fc14135cc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") "
Jul 23 04:02:26.056679 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.056590    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume\") pod \"efb84556-21ef-4b53-ad32-768fc14135cc\" (UID: \"efb84556-21ef-4b53-ad32-768fc14135cc\") "
Jul 23 04:02:26.056718 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.056700    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wptq9\" (UniqueName: \"kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9\") pod \"892b896e-f397-4ad1-bafb-4b2eec73aaa5\" (UID: \"892b896e-f397-4ad1-bafb-4b2eec73aaa5\") "
Jul 23 04:02:26.057011 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.056986    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "efb84556-21ef-4b53-ad32-768fc14135cc" (UID: "efb84556-21ef-4b53-ad32-768fc14135cc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:02:26.058480 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.058460    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22" (OuterVolumeSpecName: "kube-api-access-wdb22") pod "efb84556-21ef-4b53-ad32-768fc14135cc" (UID: "efb84556-21ef-4b53-ad32-768fc14135cc"). InnerVolumeSpecName "kube-api-access-wdb22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:02:26.058589 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.058570    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9" (OuterVolumeSpecName: "kube-api-access-wptq9") pod "892b896e-f397-4ad1-bafb-4b2eec73aaa5" (UID: "892b896e-f397-4ad1-bafb-4b2eec73aaa5"). InnerVolumeSpecName "kube-api-access-wptq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:02:26.085414 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.085396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:26.157922 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.157853    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wptq9\" (UniqueName: \"kubernetes.io/projected/892b896e-f397-4ad1-bafb-4b2eec73aaa5-kube-api-access-wptq9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:26.157922 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.157885    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdb22\" (UniqueName: \"kubernetes.io/projected/efb84556-21ef-4b53-ad32-768fc14135cc-kube-api-access-wdb22\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:26.157922 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.157899    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efb84556-21ef-4b53-ad32-768fc14135cc-extensions-socket-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:26.224416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.224395    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:26.226856 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:02:26.226830    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306232a0_eee8_41c8_aae4_60a1d3db4f71.slice/crio-4a06a3a22a6b37f2d4f1299c7e97195f51375aef03b1b94182edd7127ae52112 WatchSource:0}: Error finding container 4a06a3a22a6b37f2d4f1299c7e97195f51375aef03b1b94182edd7127ae52112: Status 404 returned error can't find the container with id 4a06a3a22a6b37f2d4f1299c7e97195f51375aef03b1b94182edd7127ae52112
Jul 23 04:02:26.963546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.963459    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" event={"ID":"306232a0-eee8-41c8-aae4-60a1d3db4f71","Type":"ContainerStarted","Data":"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"}
Jul 23 04:02:26.963990 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.963555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" event={"ID":"306232a0-eee8-41c8-aae4-60a1d3db4f71","Type":"ContainerStarted","Data":"4a06a3a22a6b37f2d4f1299c7e97195f51375aef03b1b94182edd7127ae52112"}
Jul 23 04:02:26.963990 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.963590    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:26.991293 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:26.991249    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" podStartSLOduration=1.9912358060000002 podStartE2EDuration="1.991235806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:26.990033861 +0000 UTC m=+429.451177533" watchObservedRunningTime="2026-07-23 04:02:26.991235806 +0000 UTC m=+429.452379477"
Jul 23 04:02:27.492241 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:27.492200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:27.701227 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:27.701189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:28.051912 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:28.051883    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" path="/var/lib/kubelet/pods/892b896e-f397-4ad1-bafb-4b2eec73aaa5/volumes"
Jul 23 04:02:28.052260 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:28.052210    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb84556-21ef-4b53-ad32-768fc14135cc" path="/var/lib/kubelet/pods/efb84556-21ef-4b53-ad32-768fc14135cc/volumes"
Jul 23 04:02:30.290215 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:30.290182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:30.392933 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:30.392896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:35.109311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:35.109268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:35.209564 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:35.209531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:37.972766 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:37.972732    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:38.741142 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:38.741111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:38.839741 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:38.839701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:46.015344 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.015235    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75665c68bb-42v6x" podUID="6e852e92-7a07-4c2d-9152-d13ece47318d" containerName="console" containerID="cri-o://f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33" gracePeriod=15
Jul 23 04:02:46.261431 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.261408    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 04:02:46.307958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.307885    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.307958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.307948    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.307969    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308003    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308029    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308055    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308083    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcrdw\" (UniqueName: \"kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw\") pod \"6e852e92-7a07-4c2d-9152-d13ece47318d\" (UID: \"6e852e92-7a07-4c2d-9152-d13ece47318d\") "
Jul 23 04:02:46.308558 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config" (OuterVolumeSpecName: "console-config") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:02:46.308644 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308558    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:02:46.308644 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308558    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:02:46.308644 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.308606    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:02:46.310138 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.310111    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw" (OuterVolumeSpecName: "kube-api-access-jcrdw") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "kube-api-access-jcrdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:02:46.310436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.310409    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:02:46.310436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.310423    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e852e92-7a07-4c2d-9152-d13ece47318d" (UID: "6e852e92-7a07-4c2d-9152-d13ece47318d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:02:46.409600 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409573    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-serving-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409600 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409597    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e852e92-7a07-4c2d-9152-d13ece47318d-console-oauth-config\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409610    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409622    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-trusted-ca-bundle\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409634    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-oauth-serving-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409648    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e852e92-7a07-4c2d-9152-d13ece47318d-console-config\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:46.409761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:46.409660    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcrdw\" (UniqueName: \"kubernetes.io/projected/6e852e92-7a07-4c2d-9152-d13ece47318d-kube-api-access-jcrdw\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:47.058503 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.058472    2582 generic.go:356] "Generic (PLEG): container finished" podID="6e852e92-7a07-4c2d-9152-d13ece47318d" containerID="f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33" exitCode=0
Jul 23 04:02:47.058908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.058518    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75665c68bb-42v6x"
Jul 23 04:02:47.058908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.058524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75665c68bb-42v6x" event={"ID":"6e852e92-7a07-4c2d-9152-d13ece47318d","Type":"ContainerDied","Data":"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"}
Jul 23 04:02:47.058908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.058623    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75665c68bb-42v6x" event={"ID":"6e852e92-7a07-4c2d-9152-d13ece47318d","Type":"ContainerDied","Data":"0424df4530bea9d3848604b4652a8f28be040d88b834b913945404e3dea1dcb3"}
Jul 23 04:02:47.058908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.058642    2582 scope.go:117] "RemoveContainer" containerID="f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"
Jul 23 04:02:47.067390 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.067372    2582 scope.go:117] "RemoveContainer" containerID="f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"
Jul 23 04:02:47.067632 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:47.067614    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33\": container with ID starting with f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33 not found: ID does not exist" containerID="f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"
Jul 23 04:02:47.067676 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.067639    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33"} err="failed to get container status \"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33\": rpc error: code = NotFound desc = could not find container \"f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33\": container with ID starting with f858e21f849d81ec8d8c6ce04860f36d2d6f72b23a464961e25b4bdf46bc6d33 not found: ID does not exist"
Jul 23 04:02:47.083935 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.083912    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 04:02:47.088819 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:47.088781    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75665c68bb-42v6x"]
Jul 23 04:02:48.052363 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:48.052332    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e852e92-7a07-4c2d-9152-d13ece47318d" path="/var/lib/kubelet/pods/6e852e92-7a07-4c2d-9152-d13ece47318d/volumes"
Jul 23 04:02:49.159334 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:49.159260    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:49.269213 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:49.269182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:52.596230 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:52.596194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:52.699916 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:52.699879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:52.989466 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:52.989435    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:52.989723 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:52.989696    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" podUID="306232a0-eee8-41c8-aae4-60a1d3db4f71" containerName="manager" containerID="cri-o://43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75" gracePeriod=10
Jul 23 04:02:53.232491 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.232463    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:53.261068 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.261017    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7rpk\" (UniqueName: \"kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk\") pod \"306232a0-eee8-41c8-aae4-60a1d3db4f71\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") "
Jul 23 04:02:53.261176 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.261089    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume\") pod \"306232a0-eee8-41c8-aae4-60a1d3db4f71\" (UID: \"306232a0-eee8-41c8-aae4-60a1d3db4f71\") "
Jul 23 04:02:53.261432 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.261412    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "306232a0-eee8-41c8-aae4-60a1d3db4f71" (UID: "306232a0-eee8-41c8-aae4-60a1d3db4f71"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:02:53.263172 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.263148    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk" (OuterVolumeSpecName: "kube-api-access-r7rpk") pod "306232a0-eee8-41c8-aae4-60a1d3db4f71" (UID: "306232a0-eee8-41c8-aae4-60a1d3db4f71"). InnerVolumeSpecName "kube-api-access-r7rpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:02:53.297975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.297948    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:02:53.298252 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298241    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" containerName="manager"
Jul 23 04:02:53.298299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298254    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" containerName="manager"
Jul 23 04:02:53.298299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298268    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306232a0-eee8-41c8-aae4-60a1d3db4f71" containerName="manager"
Jul 23 04:02:53.298299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298273    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="306232a0-eee8-41c8-aae4-60a1d3db4f71" containerName="manager"
Jul 23 04:02:53.298299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298288    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e852e92-7a07-4c2d-9152-d13ece47318d" containerName="console"
Jul 23 04:02:53.298299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298294    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e852e92-7a07-4c2d-9152-d13ece47318d" containerName="console"
Jul 23 04:02:53.298462 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298342    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="306232a0-eee8-41c8-aae4-60a1d3db4f71" containerName="manager"
Jul 23 04:02:53.298462 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298349    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e852e92-7a07-4c2d-9152-d13ece47318d" containerName="console"
Jul 23 04:02:53.298462 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.298357    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="892b896e-f397-4ad1-bafb-4b2eec73aaa5" containerName="manager"
Jul 23 04:02:53.301208 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.301192    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.322165 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.322143    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:02:53.362393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.362366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.362498 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.362429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9mpj\" (UniqueName: \"kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.362539 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.362513    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/306232a0-eee8-41c8-aae4-60a1d3db4f71-extensions-socket-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:53.362539 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.362532    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7rpk\" (UniqueName: \"kubernetes.io/projected/306232a0-eee8-41c8-aae4-60a1d3db4f71-kube-api-access-r7rpk\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:02:53.463597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.463569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.463713 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.463622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9mpj\" (UniqueName: \"kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.463964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.463946    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.479951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.479929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9mpj\" (UniqueName: \"kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj\") pod \"kuadrant-operator-controller-manager-86878c769f-vfsfk\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.610554 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.610493    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:53.774229 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:53.772790    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:02:53.776334 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:02:53.776309    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561d5998_2680_4219_9468_214ae159729e.slice/crio-3bbaebc9919541c7d718efc97b40bb6fe3382ca5ff40c98cc0ff7f5c62a81b3f WatchSource:0}: Error finding container 3bbaebc9919541c7d718efc97b40bb6fe3382ca5ff40c98cc0ff7f5c62a81b3f: Status 404 returned error can't find the container with id 3bbaebc9919541c7d718efc97b40bb6fe3382ca5ff40c98cc0ff7f5c62a81b3f
Jul 23 04:02:54.092110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.092071    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" event={"ID":"561d5998-2680-4219-9468-214ae159729e","Type":"ContainerStarted","Data":"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"}
Jul 23 04:02:54.092258 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.092238    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:02:54.092341 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.092286    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" event={"ID":"561d5998-2680-4219-9468-214ae159729e","Type":"ContainerStarted","Data":"3bbaebc9919541c7d718efc97b40bb6fe3382ca5ff40c98cc0ff7f5c62a81b3f"}
Jul 23 04:02:54.093307 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.093278    2582 generic.go:356] "Generic (PLEG): container finished" podID="306232a0-eee8-41c8-aae4-60a1d3db4f71" containerID="43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75" exitCode=0
Jul 23 04:02:54.093416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.093327    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"
Jul 23 04:02:54.093416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.093324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" event={"ID":"306232a0-eee8-41c8-aae4-60a1d3db4f71","Type":"ContainerDied","Data":"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"}
Jul 23 04:02:54.093416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.093375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv" event={"ID":"306232a0-eee8-41c8-aae4-60a1d3db4f71","Type":"ContainerDied","Data":"4a06a3a22a6b37f2d4f1299c7e97195f51375aef03b1b94182edd7127ae52112"}
Jul 23 04:02:54.093416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.093398    2582 scope.go:117] "RemoveContainer" containerID="43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"
Jul 23 04:02:54.101863 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.101849    2582 scope.go:117] "RemoveContainer" containerID="43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"
Jul 23 04:02:54.102080 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:02:54.102064    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75\": container with ID starting with 43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75 not found: ID does not exist" containerID="43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"
Jul 23 04:02:54.102124 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.102086    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75"} err="failed to get container status \"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75\": rpc error: code = NotFound desc = could not find container \"43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75\": container with ID starting with 43ad60715c36db22258e5dd2b36a8e6f5bdca41c07373ff0ef7cceb8aa41ce75 not found: ID does not exist"
Jul 23 04:02:54.115098 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.115060    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" podStartSLOduration=1.115046953 podStartE2EDuration="1.115046953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:02:54.112949914 +0000 UTC m=+456.574093585" watchObservedRunningTime="2026-07-23 04:02:54.115046953 +0000 UTC m=+456.576190624"
Jul 23 04:02:54.137550 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.137502    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:54.140521 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:54.140481    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmpbv"]
Jul 23 04:02:56.052207 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:56.052178    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306232a0-eee8-41c8-aae4-60a1d3db4f71" path="/var/lib/kubelet/pods/306232a0-eee8-41c8-aae4-60a1d3db4f71/volumes"
Jul 23 04:02:57.148747 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:57.148718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:57.496930 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:57.496897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:02:57.915642 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:02:57.915563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:01.124435 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:01.124402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:01.225196 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:01.225138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:05.102148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:05.102124    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:03:09.469492 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.469460    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"]
Jul 23 04:03:09.473968 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.473949    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.477090 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.477069    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xhw9l\""
Jul 23 04:03:09.486100 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.486078    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"]
Jul 23 04:03:09.498978 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.498948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:09.584519 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198d3833-107a-4c9f-9dc4-10df2acee80d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584675 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584675 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584675 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584798 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584684    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584798 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584798 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw7wm\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-kube-api-access-tw7wm\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584798 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.584798 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.584792    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.630856 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.630823    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:09.685975 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.685951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686068 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.685985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686068 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686068 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw7wm\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-kube-api-access-tw7wm\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686409 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198d3833-107a-4c9f-9dc4-10df2acee80d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686409 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686520 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686497    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686578 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686751 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.686991 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.686971    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198d3833-107a-4c9f-9dc4-10df2acee80d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.688393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.688377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.688524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.688507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.695206 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.695185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.695619 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.695599    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw7wm\" (UniqueName: \"kubernetes.io/projected/198d3833-107a-4c9f-9dc4-10df2acee80d-kube-api-access-tw7wm\") pod \"maas-default-gateway-openshift-default-56654dcf7-lhgxx\" (UID: \"198d3833-107a-4c9f-9dc4-10df2acee80d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.784344 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.784325    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:09.917558 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:09.917534    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"]
Jul 23 04:03:09.919487 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:03:09.919457    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198d3833_107a_4c9f_9dc4_10df2acee80d.slice/crio-e977bfc8f85a30ce1890048b2f46aee17f6c58a83553871034a15a2c306d00de WatchSource:0}: Error finding container e977bfc8f85a30ce1890048b2f46aee17f6c58a83553871034a15a2c306d00de: Status 404 returned error can't find the container with id e977bfc8f85a30ce1890048b2f46aee17f6c58a83553871034a15a2c306d00de
Jul 23 04:03:10.168220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:10.168148    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" event={"ID":"198d3833-107a-4c9f-9dc4-10df2acee80d","Type":"ContainerStarted","Data":"e977bfc8f85a30ce1890048b2f46aee17f6c58a83553871034a15a2c306d00de"}
Jul 23 04:03:12.482403 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:12.482367    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:03:12.482679 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:12.482435    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:03:12.482679 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:12.482464    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:03:13.185027 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:13.184988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" event={"ID":"198d3833-107a-4c9f-9dc4-10df2acee80d","Type":"ContainerStarted","Data":"05d80e3edc2f49c95618e73dca6410412f68078c6ea231a8f123f0001e2e287e"}
Jul 23 04:03:13.207934 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:13.207881    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" podStartSLOduration=1.647147313 podStartE2EDuration="4.207862728s" totalImagesPullingTime="2.560715415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:13.206342386 +0000 UTC m=+475.667486069" watchObservedRunningTime="2026-07-23 04:03:13.207862728 +0000 UTC m=+475.669006401"
Jul 23 04:03:13.784544 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:13.784508    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:13.789369 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:13.789348    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:14.189839 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.189744    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:14.190641 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.190624    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:03:14.268021 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.267988    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:14.271467 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.271443    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.273177 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.273153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:14.276425 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.276408    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pbwj7\""
Jul 23 04:03:14.276637 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.276618    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 04:03:14.292854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.292832    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:14.325661 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.325631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.325786 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.325721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8fb\" (UniqueName: \"kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.363002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.362974    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:14.399710 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.399687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:14.426585 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.426557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.426693 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.426596    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8fb\" (UniqueName: \"kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.427240 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.427220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.437486 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.437463    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8fb\" (UniqueName: \"kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb\") pod \"limitador-limitador-64c94fd64f-g9bbs\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.514759 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.514728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:14.584471 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.584443    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:14.722671 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:14.722644    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:14.725190 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:03:14.725164    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ad210f_b0f7_4e67_92d8_55a240c5c583.slice/crio-9939136fccc7eb210aad37ce18e91fdd1faffe9e2ba9aa328d701bc952b1746e WatchSource:0}: Error finding container 9939136fccc7eb210aad37ce18e91fdd1faffe9e2ba9aa328d701bc952b1746e: Status 404 returned error can't find the container with id 9939136fccc7eb210aad37ce18e91fdd1faffe9e2ba9aa328d701bc952b1746e
Jul 23 04:03:15.195709 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:15.195672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" event={"ID":"61ad210f-b0f7-4e67-92d8-55a240c5c583","Type":"ContainerStarted","Data":"9939136fccc7eb210aad37ce18e91fdd1faffe9e2ba9aa328d701bc952b1746e"}
Jul 23 04:03:15.265171 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:15.265146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:15.363971 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:15.363942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:18.213303 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:18.213246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" event={"ID":"61ad210f-b0f7-4e67-92d8-55a240c5c583","Type":"ContainerStarted","Data":"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"}
Jul 23 04:03:18.213790 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:18.213345    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:18.230735 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:18.230664    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" podStartSLOduration=1.4394545 podStartE2EDuration="4.230648274s" totalImagesPullingTime="2.791193774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:18.229000738 +0000 UTC m=+480.690144409" watchObservedRunningTime="2026-07-23 04:03:18.230648274 +0000 UTC m=+480.691791981"
Jul 23 04:03:23.633192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:23.633142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:23.734089 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:23.734050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:27.928539 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:27.928508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:28.029085 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:28.029049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:29.142071 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.142031    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:29.142532 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.142275    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" podUID="61ad210f-b0f7-4e67-92d8-55a240c5c583" containerName="limitador" containerID="cri-o://3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b" gracePeriod=30
Jul 23 04:03:29.142940 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.142910    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:29.685608 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.685587    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:29.847092 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.847069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl8fb\" (UniqueName: \"kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb\") pod \"61ad210f-b0f7-4e67-92d8-55a240c5c583\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") "
Jul 23 04:03:29.847251 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.847122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file\") pod \"61ad210f-b0f7-4e67-92d8-55a240c5c583\" (UID: \"61ad210f-b0f7-4e67-92d8-55a240c5c583\") "
Jul 23 04:03:29.847486 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.847454    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file" (OuterVolumeSpecName: "config-file") pod "61ad210f-b0f7-4e67-92d8-55a240c5c583" (UID: "61ad210f-b0f7-4e67-92d8-55a240c5c583"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:03:29.849089 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.849068    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb" (OuterVolumeSpecName: "kube-api-access-kl8fb") pod "61ad210f-b0f7-4e67-92d8-55a240c5c583" (UID: "61ad210f-b0f7-4e67-92d8-55a240c5c583"). InnerVolumeSpecName "kube-api-access-kl8fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:03:29.948032 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.948012    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl8fb\" (UniqueName: \"kubernetes.io/projected/61ad210f-b0f7-4e67-92d8-55a240c5c583-kube-api-access-kl8fb\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:03:29.948032 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:29.948032    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61ad210f-b0f7-4e67-92d8-55a240c5c583-config-file\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:03:30.264123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.264092    2582 generic.go:356] "Generic (PLEG): container finished" podID="61ad210f-b0f7-4e67-92d8-55a240c5c583" containerID="3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b" exitCode=0
Jul 23 04:03:30.264524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.264130    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"
Jul 23 04:03:30.264524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.264130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" event={"ID":"61ad210f-b0f7-4e67-92d8-55a240c5c583","Type":"ContainerDied","Data":"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"}
Jul 23 04:03:30.264524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.264258    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g9bbs" event={"ID":"61ad210f-b0f7-4e67-92d8-55a240c5c583","Type":"ContainerDied","Data":"9939136fccc7eb210aad37ce18e91fdd1faffe9e2ba9aa328d701bc952b1746e"}
Jul 23 04:03:30.264524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.264285    2582 scope.go:117] "RemoveContainer" containerID="3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"
Jul 23 04:03:30.272373 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.272359    2582 scope.go:117] "RemoveContainer" containerID="3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"
Jul 23 04:03:30.272622 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:03:30.272600    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b\": container with ID starting with 3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b not found: ID does not exist" containerID="3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"
Jul 23 04:03:30.272679 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.272630    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b"} err="failed to get container status \"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b\": rpc error: code = NotFound desc = could not find container \"3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b\": container with ID starting with 3eed6ab2a739ec9d3fd08f8a20783197e9b30dfdcf18b5519a2e13bc0300aa7b not found: ID does not exist"
Jul 23 04:03:30.281123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.281102    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:30.287120 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.287099    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g9bbs"]
Jul 23 04:03:30.385015 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.384987    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:30.475235 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.475208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:30.862553 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.862523    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"]
Jul 23 04:03:30.862875 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.862862    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ad210f-b0f7-4e67-92d8-55a240c5c583" containerName="limitador"
Jul 23 04:03:30.862928 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.862876    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ad210f-b0f7-4e67-92d8-55a240c5c583" containerName="limitador"
Jul 23 04:03:30.862970 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.862937    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ad210f-b0f7-4e67-92d8-55a240c5c583" containerName="limitador"
Jul 23 04:03:30.868060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.868042    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:30.871900 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.871849    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 04:03:30.871900 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.871854    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bsrwt\""
Jul 23 04:03:30.872660 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.872642    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 04:03:30.872765 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.872645    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 23 04:03:30.884329 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.884311    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"]
Jul 23 04:03:30.916370 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.916342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:30.955775 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.955750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af113fbf-f451-42a0-ab57-ea807e0895b9-data\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:30.955904 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:30.955824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqfgp\" (UniqueName: \"kubernetes.io/projected/af113fbf-f451-42a0-ab57-ea807e0895b9-kube-api-access-bqfgp\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.057226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.057177    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqfgp\" (UniqueName: \"kubernetes.io/projected/af113fbf-f451-42a0-ab57-ea807e0895b9-kube-api-access-bqfgp\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.057393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.057290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af113fbf-f451-42a0-ab57-ea807e0895b9-data\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.057636 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.057619    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af113fbf-f451-42a0-ab57-ea807e0895b9-data\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.066684 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.066660    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqfgp\" (UniqueName: \"kubernetes.io/projected/af113fbf-f451-42a0-ab57-ea807e0895b9-kube-api-access-bqfgp\") pod \"postgres-69fb4594cf-jhql7\" (UID: \"af113fbf-f451-42a0-ab57-ea807e0895b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.178004 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.177933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:31.307705 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:31.307681    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"]
Jul 23 04:03:31.309459 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:03:31.309431    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf113fbf_f451_42a0_ab57_ea807e0895b9.slice/crio-e7b905aa8c346aacfbdecaad21a157ed226c81506dc1e90d93b1c98596040a22 WatchSource:0}: Error finding container e7b905aa8c346aacfbdecaad21a157ed226c81506dc1e90d93b1c98596040a22: Status 404 returned error can't find the container with id e7b905aa8c346aacfbdecaad21a157ed226c81506dc1e90d93b1c98596040a22
Jul 23 04:03:32.057829 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:32.057297    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ad210f-b0f7-4e67-92d8-55a240c5c583" path="/var/lib/kubelet/pods/61ad210f-b0f7-4e67-92d8-55a240c5c583/volumes"
Jul 23 04:03:32.281524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:32.281450    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7" event={"ID":"af113fbf-f451-42a0-ab57-ea807e0895b9","Type":"ContainerStarted","Data":"e7b905aa8c346aacfbdecaad21a157ed226c81506dc1e90d93b1c98596040a22"}
Jul 23 04:03:36.301609 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:36.301572    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7" event={"ID":"af113fbf-f451-42a0-ab57-ea807e0895b9","Type":"ContainerStarted","Data":"7df2e0d01d1f297bc727f0918558eee6a7859a2126dffa35de604d9566ee1c0d"}
Jul 23 04:03:36.301964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:36.301697    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:36.325116 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:36.325036    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7" podStartSLOduration=1.564276781 podStartE2EDuration="6.325023262s" totalImagesPullingTime="4.760746481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:36.32218051 +0000 UTC m=+498.783324184" watchObservedRunningTime="2026-07-23 04:03:36.325023262 +0000 UTC m=+498.786166934"
Jul 23 04:03:42.335034 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:42.335004    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jhql7"
Jul 23 04:03:42.658192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:42.658085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:43.452613 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:43.452574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:49.671639 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.671582    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:49.686950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.686919    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:03:49.690917 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.690898    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.693598 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.693577    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h2r79\""
Jul 23 04:03:49.693706 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.693627    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 04:03:49.694525 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.694509    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 04:03:49.703433 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.703413    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:03:49.707977 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.707956    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q2sh\" (UniqueName: \"kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.708074 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.707994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.788321 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.788287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:49.809323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.809295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q2sh\" (UniqueName: \"kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.809444 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.809332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.811691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.811671    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.818530 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.818513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q2sh\" (UniqueName: \"kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh\") pod \"maas-controller-7b6f65bfdf-7j2fn\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") " pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:49.850059 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.850032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:49.889350 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.889328    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:03:49.892551 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.892535    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:49.907428 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.907407    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:03:49.909942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.909899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:49.910051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:49.910018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcn6z\" (UniqueName: \"kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.001314 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.001296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:50.011688 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.011553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.011688 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.011606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcn6z\" (UniqueName: \"kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.013750 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.013727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.021400 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.021375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcn6z\" (UniqueName: \"kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z\") pod \"maas-controller-b76d45f8-thc47\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") " pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.138450 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.138426    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:03:50.140609 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:03:50.140583    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851676de_5027_416f_ba30_d0e1c73893bf.slice/crio-c0fac3dac0673beac9ec68531a60cb074dcf4884ec9da0a479e95cfcb1b5b46e WatchSource:0}: Error finding container c0fac3dac0673beac9ec68531a60cb074dcf4884ec9da0a479e95cfcb1b5b46e: Status 404 returned error can't find the container with id c0fac3dac0673beac9ec68531a60cb074dcf4884ec9da0a479e95cfcb1b5b46e
Jul 23 04:03:50.201757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.201735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:50.335018 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.334993    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:03:50.336727 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:03:50.336700    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5018435a_9941_43e9_b824_061b29bb4efb.slice/crio-0f48f4227f2362c454de5aeccc3f969b303dd75342f01042b1befd86f78a3c45 WatchSource:0}: Error finding container 0f48f4227f2362c454de5aeccc3f969b303dd75342f01042b1befd86f78a3c45: Status 404 returned error can't find the container with id 0f48f4227f2362c454de5aeccc3f969b303dd75342f01042b1befd86f78a3c45
Jul 23 04:03:50.364480 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.364447    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" event={"ID":"851676de-5027-416f-ba30-d0e1c73893bf","Type":"ContainerStarted","Data":"c0fac3dac0673beac9ec68531a60cb074dcf4884ec9da0a479e95cfcb1b5b46e"}
Jul 23 04:03:50.365547 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:50.365525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-thc47" event={"ID":"5018435a-9941-43e9-b824-061b29bb4efb","Type":"ContainerStarted","Data":"0f48f4227f2362c454de5aeccc3f969b303dd75342f01042b1befd86f78a3c45"}
Jul 23 04:03:52.872844 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:52.872775    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:52.974248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:52.974173    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:03:53.384595 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.384560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-thc47" event={"ID":"5018435a-9941-43e9-b824-061b29bb4efb","Type":"ContainerStarted","Data":"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"}
Jul 23 04:03:53.384815 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.384628    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:03:53.386083 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.386056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" event={"ID":"851676de-5027-416f-ba30-d0e1c73893bf","Type":"ContainerStarted","Data":"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"}
Jul 23 04:03:53.386172 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.386131    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:03:53.407367 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.407322    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b76d45f8-thc47" podStartSLOduration=2.034159053 podStartE2EDuration="4.407308324s" totalImagesPullingTime="2.373149271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:53.404859935 +0000 UTC m=+515.866003624" watchObservedRunningTime="2026-07-23 04:03:53.407308324 +0000 UTC m=+515.868451990"
Jul 23 04:03:53.431541 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:03:53.431498    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" podStartSLOduration=1.868777212 podStartE2EDuration="4.431473455s" totalImagesPullingTime="2.562696243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:03:53.429723178 +0000 UTC m=+515.890866851" watchObservedRunningTime="2026-07-23 04:03:53.431473455 +0000 UTC m=+515.892617205"
Jul 23 04:04:04.392504 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.392470    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:04:04.392977 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.392548    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:04:04.456232 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.456201    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:04:04.456406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.456384    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" podUID="851676de-5027-416f-ba30-d0e1c73893bf" containerName="manager" containerID="cri-o://466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598" gracePeriod=30
Jul 23 04:04:04.702329 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.702304    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:04:04.829108 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.829071    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert\") pod \"851676de-5027-416f-ba30-d0e1c73893bf\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") "
Jul 23 04:04:04.829289 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.829129    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q2sh\" (UniqueName: \"kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh\") pod \"851676de-5027-416f-ba30-d0e1c73893bf\" (UID: \"851676de-5027-416f-ba30-d0e1c73893bf\") "
Jul 23 04:04:04.831176 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.831153    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "851676de-5027-416f-ba30-d0e1c73893bf" (UID: "851676de-5027-416f-ba30-d0e1c73893bf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:04:04.831314 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.831293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh" (OuterVolumeSpecName: "kube-api-access-7q2sh") pod "851676de-5027-416f-ba30-d0e1c73893bf" (UID: "851676de-5027-416f-ba30-d0e1c73893bf"). InnerVolumeSpecName "kube-api-access-7q2sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:04:04.930199 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.930172    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/851676de-5027-416f-ba30-d0e1c73893bf-webhook-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:04:04.930199 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:04.930195    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q2sh\" (UniqueName: \"kubernetes.io/projected/851676de-5027-416f-ba30-d0e1c73893bf-kube-api-access-7q2sh\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:04:05.441084 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.441048    2582 generic.go:356] "Generic (PLEG): container finished" podID="851676de-5027-416f-ba30-d0e1c73893bf" containerID="466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598" exitCode=0
Jul 23 04:04:05.441084 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.441088    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn"
Jul 23 04:04:05.441509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.441130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" event={"ID":"851676de-5027-416f-ba30-d0e1c73893bf","Type":"ContainerDied","Data":"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"}
Jul 23 04:04:05.441509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.441181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6f65bfdf-7j2fn" event={"ID":"851676de-5027-416f-ba30-d0e1c73893bf","Type":"ContainerDied","Data":"c0fac3dac0673beac9ec68531a60cb074dcf4884ec9da0a479e95cfcb1b5b46e"}
Jul 23 04:04:05.441509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.441204    2582 scope.go:117] "RemoveContainer" containerID="466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"
Jul 23 04:04:05.449787 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.449770    2582 scope.go:117] "RemoveContainer" containerID="466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"
Jul 23 04:04:05.450059 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:04:05.450041    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598\": container with ID starting with 466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598 not found: ID does not exist" containerID="466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"
Jul 23 04:04:05.450106 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.450067    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598"} err="failed to get container status \"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598\": rpc error: code = NotFound desc = could not find container \"466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598\": container with ID starting with 466724b877c5549290e8e3d7b782f17b148ac1b7bb6cce207f7435efeb461598 not found: ID does not exist"
Jul 23 04:04:05.462434 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.462414    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:04:05.466866 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:05.466846    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b6f65bfdf-7j2fn"]
Jul 23 04:04:06.052025 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:06.051988    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="851676de-5027-416f-ba30-d0e1c73893bf" path="/var/lib/kubelet/pods/851676de-5027-416f-ba30-d0e1c73893bf/volumes"
Jul 23 04:04:18.691931 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:18.691894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:18.791852 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:18.791795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:24.576356 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:24.576316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:24.747090 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:24.747057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:25.238702 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.238662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:25.281889 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.281858    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"]
Jul 23 04:04:25.282359 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.282345    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851676de-5027-416f-ba30-d0e1c73893bf" containerName="manager"
Jul 23 04:04:25.282408 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.282362    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="851676de-5027-416f-ba30-d0e1c73893bf" containerName="manager"
Jul 23 04:04:25.282445 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.282439    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="851676de-5027-416f-ba30-d0e1c73893bf" containerName="manager"
Jul 23 04:04:25.290504 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.290483    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.294148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.294126    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 23 04:04:25.294313 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.294284    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhj9q\""
Jul 23 04:04:25.299634 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.299613    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"]
Jul 23 04:04:25.391312 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.391277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-openshift-service-ca\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.391472 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.391327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.391472 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.391391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sbwk\" (UniqueName: \"kubernetes.io/projected/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-kube-api-access-4sbwk\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.492045 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.491961    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sbwk\" (UniqueName: \"kubernetes.io/projected/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-kube-api-access-4sbwk\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.492317 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.492076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-openshift-service-ca\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.492317 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.492113    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.492317 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:04:25.492235    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 23 04:04:25.492317 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:04:25.492298    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls podName:34edca31-e28c-49a9-b1a2-ae1bd5933ddb nodeName:}" failed. No retries permitted until 2026-07-23 04:04:25.992276786 +0000 UTC m=+548.453420442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls") pod "maas-api-5cd75f5b7c-khrhj" (UID: "34edca31-e28c-49a9-b1a2-ae1bd5933ddb") : secret "maas-api-serving-cert" not found
Jul 23 04:04:25.492892 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.492871    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-openshift-service-ca\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.503703 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.503677    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sbwk\" (UniqueName: \"kubernetes.io/projected/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-kube-api-access-4sbwk\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.590448 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.590414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:25.694178 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.694133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:25.799768 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.799683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:25.800899 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.800877    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"]
Jul 23 04:04:25.804947 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.804928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.807538 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.807522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 04:04:25.815717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.815693    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"]
Jul 23 04:04:25.831427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.831402    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-srghr"]
Jul 23 04:04:25.835638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.835620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:25.840189 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.840172    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bdsrl\""
Jul 23 04:04:25.848515 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.848495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-srghr"]
Jul 23 04:04:25.894855 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.894826    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d512474f-d91b-4cc8-85ac-394d6580a0ef-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.894984 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.894965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjhd6\" (UniqueName: \"kubernetes.io/projected/d512474f-d91b-4cc8-85ac-394d6580a0ef-kube-api-access-wjhd6\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.995585 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.995548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjhd6\" (UniqueName: \"kubernetes.io/projected/d512474f-d91b-4cc8-85ac-394d6580a0ef-kube-api-access-wjhd6\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.995585 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.995586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqn6\" (UniqueName: \"kubernetes.io/projected/96cdee91-9254-4db2-a07d-d4eef07df505-kube-api-access-djqn6\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:25.995778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.995617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d512474f-d91b-4cc8-85ac-394d6580a0ef-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.995778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.995646    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:25.995778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.995677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96cdee91-9254-4db2-a07d-d4eef07df505-plugins-config-volume\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:25.996294 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.996270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d512474f-d91b-4cc8-85ac-394d6580a0ef-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:25.998191 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:25.998162    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34edca31-e28c-49a9-b1a2-ae1bd5933ddb-maas-api-tls\") pod \"maas-api-5cd75f5b7c-khrhj\" (UID: \"34edca31-e28c-49a9-b1a2-ae1bd5933ddb\") " pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:26.003512 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.003491    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjhd6\" (UniqueName: \"kubernetes.io/projected/d512474f-d91b-4cc8-85ac-394d6580a0ef-kube-api-access-wjhd6\") pod \"payload-pre-processing-7677b774bb-kpqs9\" (UID: \"d512474f-d91b-4cc8-85ac-394d6580a0ef\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:26.096463 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.096383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96cdee91-9254-4db2-a07d-d4eef07df505-plugins-config-volume\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:26.096595 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.096471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqn6\" (UniqueName: \"kubernetes.io/projected/96cdee91-9254-4db2-a07d-d4eef07df505-kube-api-access-djqn6\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:26.097118 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.097099    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96cdee91-9254-4db2-a07d-d4eef07df505-plugins-config-volume\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:26.105046 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.105019    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqn6\" (UniqueName: \"kubernetes.io/projected/96cdee91-9254-4db2-a07d-d4eef07df505-kube-api-access-djqn6\") pod \"payload-processing-5bf47cd988-srghr\" (UID: \"96cdee91-9254-4db2-a07d-d4eef07df505\") " pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:26.115763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.115746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:26.145771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.145743    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:26.203063 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.202477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:26.275878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.275380    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"]
Jul 23 04:04:26.279363 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:04:26.279312    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd512474f_d91b_4cc8_85ac_394d6580a0ef.slice/crio-a86fd6f74e1dc4ffee4b67af4df8aeb6d2218b7174cd79dcf5b7235e6bb15017 WatchSource:0}: Error finding container a86fd6f74e1dc4ffee4b67af4df8aeb6d2218b7174cd79dcf5b7235e6bb15017: Status 404 returned error can't find the container with id a86fd6f74e1dc4ffee4b67af4df8aeb6d2218b7174cd79dcf5b7235e6bb15017
Jul 23 04:04:26.367280 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.367129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"]
Jul 23 04:04:26.374017 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:04:26.373919    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34edca31_e28c_49a9_b1a2_ae1bd5933ddb.slice/crio-9c77a472b1696f74ba702b683440dec5da968ad04bbd3c0ec8fbf8440aa986d4 WatchSource:0}: Error finding container 9c77a472b1696f74ba702b683440dec5da968ad04bbd3c0ec8fbf8440aa986d4: Status 404 returned error can't find the container with id 9c77a472b1696f74ba702b683440dec5da968ad04bbd3c0ec8fbf8440aa986d4
Jul 23 04:04:26.523306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.523124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-srghr"]
Jul 23 04:04:26.525907 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:04:26.525874    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96cdee91_9254_4db2_a07d_d4eef07df505.slice/crio-6f38ecbb73157bceedee1249657c609197552a425282ebde35aa393e05be323b WatchSource:0}: Error finding container 6f38ecbb73157bceedee1249657c609197552a425282ebde35aa393e05be323b: Status 404 returned error can't find the container with id 6f38ecbb73157bceedee1249657c609197552a425282ebde35aa393e05be323b
Jul 23 04:04:26.544015 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.543903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj" event={"ID":"34edca31-e28c-49a9-b1a2-ae1bd5933ddb","Type":"ContainerStarted","Data":"9c77a472b1696f74ba702b683440dec5da968ad04bbd3c0ec8fbf8440aa986d4"}
Jul 23 04:04:26.545387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.545324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9" event={"ID":"d512474f-d91b-4cc8-85ac-394d6580a0ef","Type":"ContainerStarted","Data":"a86fd6f74e1dc4ffee4b67af4df8aeb6d2218b7174cd79dcf5b7235e6bb15017"}
Jul 23 04:04:26.546854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:26.546828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-srghr" event={"ID":"96cdee91-9254-4db2-a07d-d4eef07df505","Type":"ContainerStarted","Data":"6f38ecbb73157bceedee1249657c609197552a425282ebde35aa393e05be323b"}
Jul 23 04:04:30.580335 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.580300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-srghr" event={"ID":"96cdee91-9254-4db2-a07d-d4eef07df505","Type":"ContainerStarted","Data":"cd31c0100a85862d110a8dfe80666f66bcb4085c425ee441f515e4679d06a1cb"}
Jul 23 04:04:30.580829 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.580371    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:30.581985 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.581958    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj" event={"ID":"34edca31-e28c-49a9-b1a2-ae1bd5933ddb","Type":"ContainerStarted","Data":"95ce5e5cd1116d37b1fe7aba74585f4a5d5725f8862f036741e1f173a1835e62"}
Jul 23 04:04:30.582107 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.582067    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:30.583635 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.583611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9" event={"ID":"d512474f-d91b-4cc8-85ac-394d6580a0ef","Type":"ContainerStarted","Data":"8a9582948b57d5a5231a21972840b8bdc0e37b60abb539d3d85545f284b855bc"}
Jul 23 04:04:30.583747 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.583663    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:30.602952 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.602913    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-srghr" podStartSLOduration=2.233398576 podStartE2EDuration="5.602899689s" totalImagesPullingTime="3.369501113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:04:30.599946449 +0000 UTC m=+553.061090124" watchObservedRunningTime="2026-07-23 04:04:30.602899689 +0000 UTC m=+553.064043360"
Jul 23 04:04:30.621533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.621490    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj" podStartSLOduration=2.168904551 podStartE2EDuration="5.621477954s" totalImagesPullingTime="3.452573403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:04:30.619129911 +0000 UTC m=+553.080273582" watchObservedRunningTime="2026-07-23 04:04:30.621477954 +0000 UTC m=+553.082621627"
Jul 23 04:04:30.637017 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:30.636969    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9" podStartSLOduration=1.997695646 podStartE2EDuration="5.63695325s" totalImagesPullingTime="3.639257604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:04:30.635306336 +0000 UTC m=+553.096450009" watchObservedRunningTime="2026-07-23 04:04:30.63695325 +0000 UTC m=+553.098096923"
Jul 23 04:04:32.187137 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:32.187070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:33.599944 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.599911    2582 generic.go:356] "Generic (PLEG): container finished" podID="198d3833-107a-4c9f-9dc4-10df2acee80d" containerID="05d80e3edc2f49c95618e73dca6410412f68078c6ea231a8f123f0001e2e287e" exitCode=0
Jul 23 04:04:33.600344 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.599982    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" event={"ID":"198d3833-107a-4c9f-9dc4-10df2acee80d","Type":"ContainerDied","Data":"05d80e3edc2f49c95618e73dca6410412f68078c6ea231a8f123f0001e2e287e"}
Jul 23 04:04:33.600386 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.600367    2582 scope.go:117] "RemoveContainer" containerID="05d80e3edc2f49c95618e73dca6410412f68078c6ea231a8f123f0001e2e287e"
Jul 23 04:04:33.601284 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.601254    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:04:33.601350 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.601322    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:04:33.601386 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:33.601353    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:04:34.606763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:34.606735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" event={"ID":"198d3833-107a-4c9f-9dc4-10df2acee80d","Type":"ContainerStarted","Data":"bbd27ee1fca213308e1eedfde90a6021d729da91ed91246a240bf21f20add56c"}
Jul 23 04:04:34.785394 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:34.785356    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:04:34.785629 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:34.785608    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lhgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 23 04:04:34.785715 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:34.785661    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" podUID="198d3833-107a-4c9f-9dc4-10df2acee80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 23 04:04:35.785345 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:35.785308    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lhgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 23 04:04:35.785731 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:35.785363    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" podUID="198d3833-107a-4c9f-9dc4-10df2acee80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 23 04:04:36.595154 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:36.595129    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cd75f5b7c-khrhj"
Jul 23 04:04:36.785280 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:36.785245    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lhgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 23 04:04:36.785445 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:36.785302    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" podUID="198d3833-107a-4c9f-9dc4-10df2acee80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 23 04:04:37.760120 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:37.760084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:37.785582 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:37.785549    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lhgxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 23 04:04:37.785960 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:37.785599    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx" podUID="198d3833-107a-4c9f-9dc4-10df2acee80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 23 04:04:37.926810 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:37.926777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:38.789080 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:38.789042    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:04:38.789550 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:38.789411    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:04:38.790405 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:38.790383    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lhgxx"
Jul 23 04:04:41.591865 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:41.591836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-srghr"
Jul 23 04:04:41.592332 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:41.591912    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-kpqs9"
Jul 23 04:04:47.511083 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.511049    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"]
Jul 23 04:04:47.515247 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.515223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.518034 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.518007    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 04:04:47.518194 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.518059    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 04:04:47.519142 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.519116    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tznxk\""
Jul 23 04:04:47.519242 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.519159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 04:04:47.525139 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.525113    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"]
Jul 23 04:04:47.547908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.547882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:04:47.605243 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.605353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06027ed6-cd34-48cf-abd8-85822b1c564c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.605353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476vl\" (UniqueName: \"kubernetes.io/projected/06027ed6-cd34-48cf-abd8-85822b1c564c-kube-api-access-476vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.605353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605297    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.605471 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.605471 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.605405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.705895 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.705866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.705895 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.705897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06027ed6-cd34-48cf-abd8-85822b1c564c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706115 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.705915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476vl\" (UniqueName: \"kubernetes.io/projected/06027ed6-cd34-48cf-abd8-85822b1c564c-kube-api-access-476vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706115 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.705933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706115 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.706016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706115 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.706049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706339 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.706323    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706397 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.706342    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.706440 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.706401    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.708231 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.708212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06027ed6-cd34-48cf-abd8-85822b1c564c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.708492 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.708473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06027ed6-cd34-48cf-abd8-85822b1c564c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.714351 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.714328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-476vl\" (UniqueName: \"kubernetes.io/projected/06027ed6-cd34-48cf-abd8-85822b1c564c-kube-api-access-476vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f\" (UID: \"06027ed6-cd34-48cf-abd8-85822b1c564c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.826649 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.826589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:47.962202 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:47.962178    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"]
Jul 23 04:04:47.963872 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:04:47.963841    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06027ed6_cd34_48cf_abd8_85822b1c564c.slice/crio-875478ad8b1bb7c74467766ca2eb2dfc9bc9566c4d8abdc15e82b1f678470f72 WatchSource:0}: Error finding container 875478ad8b1bb7c74467766ca2eb2dfc9bc9566c4d8abdc15e82b1f678470f72: Status 404 returned error can't find the container with id 875478ad8b1bb7c74467766ca2eb2dfc9bc9566c4d8abdc15e82b1f678470f72
Jul 23 04:04:48.672479 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:48.672446    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f" event={"ID":"06027ed6-cd34-48cf-abd8-85822b1c564c","Type":"ContainerStarted","Data":"875478ad8b1bb7c74467766ca2eb2dfc9bc9566c4d8abdc15e82b1f678470f72"}
Jul 23 04:04:49.678446 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:49.678408    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f" event={"ID":"06027ed6-cd34-48cf-abd8-85822b1c564c","Type":"ContainerStarted","Data":"01f5f08a4ab8295bc10c4c1dce65daad3d5bf3d9d1733c2b95444bb9cccc027d"}
Jul 23 04:04:49.697628 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:49.697580    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f" podStartSLOduration=1.486800911 podStartE2EDuration="2.697564805s" totalImagesPullingTime="1.210763894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:04:49.697008711 +0000 UTC m=+572.158152405" watchObservedRunningTime="2026-07-23 04:04:49.697564805 +0000 UTC m=+572.158708474"
Jul 23 04:04:57.827280 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:57.827246    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:57.827693 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:57.827335    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:57.839654 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:57.839632    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:04:58.729930 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:04:58.729895    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f"
Jul 23 04:05:01.437864 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:01.437830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:05:11.348073 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:11.348033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:05:23.304166 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.304129    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"]
Jul 23 04:05:23.310282 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.310259    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.313105 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.313082    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 23 04:05:23.323292 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.323270    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"]
Jul 23 04:05:23.342199 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.342176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:05:23.386722 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.386878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.386878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.387008 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.387008 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386904    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.387008 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.386982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5z7j\" (UniqueName: \"kubernetes.io/projected/1390e177-42ed-40f0-ba86-fd3bbb84df0f-kube-api-access-c5z7j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5z7j\" (UniqueName: \"kubernetes.io/projected/1390e177-42ed-40f0-ba86-fd3bbb84df0f-kube-api-access-c5z7j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488503 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488562 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488531    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.488618 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.488580    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.490370 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.490347    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1390e177-42ed-40f0-ba86-fd3bbb84df0f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.490617 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.490603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1390e177-42ed-40f0-ba86-fd3bbb84df0f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.497156 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.497133    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5z7j\" (UniqueName: \"kubernetes.io/projected/1390e177-42ed-40f0-ba86-fd3bbb84df0f-kube-api-access-c5z7j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt\" (UID: \"1390e177-42ed-40f0-ba86-fd3bbb84df0f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.621620 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.621564    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:23.755011 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.754988    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"]
Jul 23 04:05:23.757197 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:05:23.757167    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1390e177_42ed_40f0_ba86_fd3bbb84df0f.slice/crio-f9898b6a5c2113fff035f781ca96d3f7484aed27971930afe42cf72849e6d728 WatchSource:0}: Error finding container f9898b6a5c2113fff035f781ca96d3f7484aed27971930afe42cf72849e6d728: Status 404 returned error can't find the container with id f9898b6a5c2113fff035f781ca96d3f7484aed27971930afe42cf72849e6d728
Jul 23 04:05:23.758468 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.758452    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:05:23.835757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:23.835724    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt" event={"ID":"1390e177-42ed-40f0-ba86-fd3bbb84df0f","Type":"ContainerStarted","Data":"f9898b6a5c2113fff035f781ca96d3f7484aed27971930afe42cf72849e6d728"}
Jul 23 04:05:24.842226 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:24.842191    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt" event={"ID":"1390e177-42ed-40f0-ba86-fd3bbb84df0f","Type":"ContainerStarted","Data":"f8fafedec6b840428dde5e1fda85e24e3dc642aaab6171919cc4411f64a01ae3"}
Jul 23 04:05:24.861985 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:24.861933    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt" podStartSLOduration=1.687057506 podStartE2EDuration="1.861918317s" totalImagesPullingTime="174.860811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:05:24.860622088 +0000 UTC m=+607.321765761" watchObservedRunningTime="2026-07-23 04:05:24.861918317 +0000 UTC m=+607.323062016"
Jul 23 04:05:28.231082 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:28.231042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:05:33.622532 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:33.622496    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:33.622920 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:33.622565    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:33.635746 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:33.635723    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:33.896026 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:33.895920    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt"
Jul 23 04:05:37.001659 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.001629    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"]
Jul 23 04:05:37.006393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.006369    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.008864 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.008842    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 04:05:37.016355 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.016332    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"]
Jul 23 04:05:37.032727 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.032688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:05:37.102411 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6254b88-a0e7-4550-846d-c29e1b4e5842-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.102552 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.102552 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2d5m\" (UniqueName: \"kubernetes.io/projected/f6254b88-a0e7-4550-846d-c29e1b4e5842-kube-api-access-t2d5m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.102680 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102545    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.102680 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.102680 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.102653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203445 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203621 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203621 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203573    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6254b88-a0e7-4550-846d-c29e1b4e5842-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203621 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203629    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2d5m\" (UniqueName: \"kubernetes.io/projected/f6254b88-a0e7-4550-846d-c29e1b4e5842-kube-api-access-t2d5m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.203950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203896    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.204019 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.203987    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.204082 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.204063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.205777 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.205760    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6254b88-a0e7-4550-846d-c29e1b4e5842-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.206084 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.206068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6254b88-a0e7-4550-846d-c29e1b4e5842-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.212040 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.212016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2d5m\" (UniqueName: \"kubernetes.io/projected/f6254b88-a0e7-4550-846d-c29e1b4e5842-kube-api-access-t2d5m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd\" (UID: \"f6254b88-a0e7-4550-846d-c29e1b4e5842\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.316717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.316637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:37.454513 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.454490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"]
Jul 23 04:05:37.456264 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:05:37.456236    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6254b88_a0e7_4550_846d_c29e1b4e5842.slice/crio-ccee7397f0250ab13dbb521098ff1c4bb911fe8e502e11c27b7de4b9af83b66d WatchSource:0}: Error finding container ccee7397f0250ab13dbb521098ff1c4bb911fe8e502e11c27b7de4b9af83b66d: Status 404 returned error can't find the container with id ccee7397f0250ab13dbb521098ff1c4bb911fe8e502e11c27b7de4b9af83b66d
Jul 23 04:05:37.909654 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.909571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd" event={"ID":"f6254b88-a0e7-4550-846d-c29e1b4e5842","Type":"ContainerStarted","Data":"df67cf2116960eb9fe2e3286350bbf7a31aebacdbb0d0d572da547326d0b9bd5"}
Jul 23 04:05:37.909654 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.909617    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd" event={"ID":"f6254b88-a0e7-4550-846d-c29e1b4e5842","Type":"ContainerStarted","Data":"ccee7397f0250ab13dbb521098ff1c4bb911fe8e502e11c27b7de4b9af83b66d"}
Jul 23 04:05:37.929292 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:37.929251    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd" podStartSLOduration=1.7593689719999999 podStartE2EDuration="1.929238651s" totalImagesPullingTime="169.869679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:05:37.926990266 +0000 UTC m=+620.388133938" watchObservedRunningTime="2026-07-23 04:05:37.929238651 +0000 UTC m=+620.390382323"
Jul 23 04:05:47.317337 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:47.317261    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:47.317337 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:47.317314    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:47.329691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:47.329672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:05:47.970681 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:05:47.970656    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd"
Jul 23 04:06:29.888862 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:29.888797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:06:29.995492 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:29.995457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:06:31.101473 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:31.101437    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:06:31.197393 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:31.197356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:06:32.769564 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:32.769523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:06:46.215880 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:06:46.215844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:07:17.743510 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:17.743465    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:07:17.744152 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:17.743704    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b76d45f8-thc47" podUID="5018435a-9941-43e9-b824-061b29bb4efb" containerName="manager" containerID="cri-o://0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2" gracePeriod=30
Jul 23 04:07:17.996676 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:17.996621    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:07:18.126851 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.126824    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcn6z\" (UniqueName: \"kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z\") pod \"5018435a-9941-43e9-b824-061b29bb4efb\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") "
Jul 23 04:07:18.127004 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.126879    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert\") pod \"5018435a-9941-43e9-b824-061b29bb4efb\" (UID: \"5018435a-9941-43e9-b824-061b29bb4efb\") "
Jul 23 04:07:18.128667 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.128634    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5018435a-9941-43e9-b824-061b29bb4efb" (UID: "5018435a-9941-43e9-b824-061b29bb4efb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:07:18.128781 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.128730    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z" (OuterVolumeSpecName: "kube-api-access-tcn6z") pod "5018435a-9941-43e9-b824-061b29bb4efb" (UID: "5018435a-9941-43e9-b824-061b29bb4efb"). InnerVolumeSpecName "kube-api-access-tcn6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:07:18.228181 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.228153    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcn6z\" (UniqueName: \"kubernetes.io/projected/5018435a-9941-43e9-b824-061b29bb4efb-kube-api-access-tcn6z\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:07:18.228181 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.228180    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5018435a-9941-43e9-b824-061b29bb4efb-webhook-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:07:18.391680 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.391604    2582 generic.go:356] "Generic (PLEG): container finished" podID="5018435a-9941-43e9-b824-061b29bb4efb" containerID="0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2" exitCode=0
Jul 23 04:07:18.391680 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.391651    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-thc47"
Jul 23 04:07:18.391874 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.391691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-thc47" event={"ID":"5018435a-9941-43e9-b824-061b29bb4efb","Type":"ContainerDied","Data":"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"}
Jul 23 04:07:18.391874 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.391742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-thc47" event={"ID":"5018435a-9941-43e9-b824-061b29bb4efb","Type":"ContainerDied","Data":"0f48f4227f2362c454de5aeccc3f969b303dd75342f01042b1befd86f78a3c45"}
Jul 23 04:07:18.391874 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.391763    2582 scope.go:117] "RemoveContainer" containerID="0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"
Jul 23 04:07:18.401353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.401332    2582 scope.go:117] "RemoveContainer" containerID="0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"
Jul 23 04:07:18.401601 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:07:18.401584    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2\": container with ID starting with 0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2 not found: ID does not exist" containerID="0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"
Jul 23 04:07:18.401652 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.401611    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2"} err="failed to get container status \"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2\": rpc error: code = NotFound desc = could not find container \"0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2\": container with ID starting with 0947a186bbaae779af2da9ed091398add014b1c43c6e66a044bda9189de23da2 not found: ID does not exist"
Jul 23 04:07:18.414121 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.414100    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:07:18.421010 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:18.420986    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b76d45f8-thc47"]
Jul 23 04:07:20.052437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:07:20.052404    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5018435a-9941-43e9-b824-061b29bb4efb" path="/var/lib/kubelet/pods/5018435a-9941-43e9-b824-061b29bb4efb/volumes"
Jul 23 04:08:06.493316 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:06.493280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:08:06.554275 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:06.554238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:08:19.256637 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.256605    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:19.257026 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.257005    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5018435a-9941-43e9-b824-061b29bb4efb" containerName="manager"
Jul 23 04:08:19.257026 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.257018    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5018435a-9941-43e9-b824-061b29bb4efb" containerName="manager"
Jul 23 04:08:19.257103 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.257076    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5018435a-9941-43e9-b824-061b29bb4efb" containerName="manager"
Jul 23 04:08:19.260445 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.260429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.263147 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.263120    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 04:08:19.264268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.264249    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 04:08:19.264373 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.264249    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h2r79\""
Jul 23 04:08:19.273198 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.273173    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:19.309542 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.309519    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4qs\" (UniqueName: \"kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.309649 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.309567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.410736 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.410695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4qs\" (UniqueName: \"kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.410939 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.410790    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.413519 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.413484    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.420883 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.420859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4qs\" (UniqueName: \"kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs\") pod \"maas-controller-b76d45f8-rvrsv\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") " pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.571195 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.571132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:19.700899 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:19.700874    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:19.702954 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:08:19.702930    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1205b8_4f87_4ebf_b6ce_b3a707acbc43.slice/crio-4b9255e65a7d86cda274acb5bc06d88b93029adcd8e34a41cddf951887d86d5e WatchSource:0}: Error finding container 4b9255e65a7d86cda274acb5bc06d88b93029adcd8e34a41cddf951887d86d5e: Status 404 returned error can't find the container with id 4b9255e65a7d86cda274acb5bc06d88b93029adcd8e34a41cddf951887d86d5e
Jul 23 04:08:20.684512 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:20.684475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-rvrsv" event={"ID":"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43","Type":"ContainerStarted","Data":"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"}
Jul 23 04:08:20.684921 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:20.684528    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-rvrsv" event={"ID":"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43","Type":"ContainerStarted","Data":"4b9255e65a7d86cda274acb5bc06d88b93029adcd8e34a41cddf951887d86d5e"}
Jul 23 04:08:20.684921 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:20.684610    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:20.719670 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:20.719604    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b76d45f8-rvrsv" podStartSLOduration=1.364432457 podStartE2EDuration="1.719590761s" totalImagesPullingTime="355.158304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:08:20.719418657 +0000 UTC m=+783.180562329" watchObservedRunningTime="2026-07-23 04:08:20.719590761 +0000 UTC m=+783.180734433"
Jul 23 04:08:31.691049 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:31.691018    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:36.245857 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.245816    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:08:36.342083 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.342051    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:36.342267 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.342239    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b76d45f8-rvrsv" podUID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" containerName="manager" containerID="cri-o://248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9" gracePeriod=30
Jul 23 04:08:36.593455 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.593432    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:36.761427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.761352    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" containerID="248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9" exitCode=0
Jul 23 04:08:36.761427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.761397    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-rvrsv"
Jul 23 04:08:36.761594 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.761433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-rvrsv" event={"ID":"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43","Type":"ContainerDied","Data":"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"}
Jul 23 04:08:36.761594 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.761485    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-rvrsv" event={"ID":"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43","Type":"ContainerDied","Data":"4b9255e65a7d86cda274acb5bc06d88b93029adcd8e34a41cddf951887d86d5e"}
Jul 23 04:08:36.761594 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.761506    2582 scope.go:117] "RemoveContainer" containerID="248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"
Jul 23 04:08:36.769686 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.769323    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c4qs\" (UniqueName: \"kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs\") pod \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") "
Jul 23 04:08:36.769686 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.769488    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert\") pod \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\" (UID: \"6c1205b8-4f87-4ebf-b6ce-b3a707acbc43\") "
Jul 23 04:08:36.770422 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.770398    2582 scope.go:117] "RemoveContainer" containerID="248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"
Jul 23 04:08:36.776642 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.771899    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs" (OuterVolumeSpecName: "kube-api-access-8c4qs") pod "6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" (UID: "6c1205b8-4f87-4ebf-b6ce-b3a707acbc43"). InnerVolumeSpecName "kube-api-access-8c4qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:08:36.777096 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.776978    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" (UID: "6c1205b8-4f87-4ebf-b6ce-b3a707acbc43"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:08:36.777096 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:08:36.776943    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9\": container with ID starting with 248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9 not found: ID does not exist" containerID="248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"
Jul 23 04:08:36.777252 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.777118    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9"} err="failed to get container status \"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9\": rpc error: code = NotFound desc = could not find container \"248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9\": container with ID starting with 248a20ebbb34c977e670d3c11713ecbbcfba7056bba36f47fb42723f811677c9 not found: ID does not exist"
Jul 23 04:08:36.870411 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.870387    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-webhook-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:08:36.870411 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:36.870412    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c4qs\" (UniqueName: \"kubernetes.io/projected/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43-kube-api-access-8c4qs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:08:37.083266 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.083240    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:37.089320 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.089294    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b76d45f8-rvrsv"]
Jul 23 04:08:37.786918 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.786873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:08:37.891607 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.891574    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b76d45f8-lz4rv"]
Jul 23 04:08:37.891966 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.891952    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" containerName="manager"
Jul 23 04:08:37.891966 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.891967    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" containerName="manager"
Jul 23 04:08:37.892094 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.892032    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" containerName="manager"
Jul 23 04:08:37.896478 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.896457    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:37.899149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.899130    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 04:08:37.899286 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.899270    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 04:08:37.900260 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.900244    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h2r79\""
Jul 23 04:08:37.907678 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:37.907657    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-lz4rv"]
Jul 23 04:08:38.053058 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.052967    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c1205b8-4f87-4ebf-b6ce-b3a707acbc43" path="/var/lib/kubelet/pods/6c1205b8-4f87-4ebf-b6ce-b3a707acbc43/volumes"
Jul 23 04:08:38.080867 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.080839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b40e973b-7433-421f-90a4-c9ef2d8253c4-webhook-cert\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.081029 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.080876    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbsdt\" (UniqueName: \"kubernetes.io/projected/b40e973b-7433-421f-90a4-c9ef2d8253c4-kube-api-access-lbsdt\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.182294 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.182265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b40e973b-7433-421f-90a4-c9ef2d8253c4-webhook-cert\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.182436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.182303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbsdt\" (UniqueName: \"kubernetes.io/projected/b40e973b-7433-421f-90a4-c9ef2d8253c4-kube-api-access-lbsdt\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.184549 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.184528    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b40e973b-7433-421f-90a4-c9ef2d8253c4-webhook-cert\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.191117 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.191089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbsdt\" (UniqueName: \"kubernetes.io/projected/b40e973b-7433-421f-90a4-c9ef2d8253c4-kube-api-access-lbsdt\") pod \"maas-controller-b76d45f8-lz4rv\" (UID: \"b40e973b-7433-421f-90a4-c9ef2d8253c4\") " pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.207495 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.207476    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.344819 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.344785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b76d45f8-lz4rv"]
Jul 23 04:08:38.347583 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:08:38.347556    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40e973b_7433_421f_90a4_c9ef2d8253c4.slice/crio-f59126633382b51cdcaef2ebd719e3b50df2d0776adb6d0aecec6451084716cb WatchSource:0}: Error finding container f59126633382b51cdcaef2ebd719e3b50df2d0776adb6d0aecec6451084716cb: Status 404 returned error can't find the container with id f59126633382b51cdcaef2ebd719e3b50df2d0776adb6d0aecec6451084716cb
Jul 23 04:08:38.775113 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.775082    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-lz4rv" event={"ID":"b40e973b-7433-421f-90a4-c9ef2d8253c4","Type":"ContainerStarted","Data":"800da4ce674a5a5442adfe4278f7fa3576dae3b39ec56e5643f1cd4fb4261ebb"}
Jul 23 04:08:38.775246 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.775135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b76d45f8-lz4rv" event={"ID":"b40e973b-7433-421f-90a4-c9ef2d8253c4","Type":"ContainerStarted","Data":"f59126633382b51cdcaef2ebd719e3b50df2d0776adb6d0aecec6451084716cb"}
Jul 23 04:08:38.775246 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.775182    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:08:38.792419 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:38.792376    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b76d45f8-lz4rv" podStartSLOduration=1.452878293 podStartE2EDuration="1.792361041s" totalImagesPullingTime="339.482748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:08:38.790851377 +0000 UTC m=+801.251995076" watchObservedRunningTime="2026-07-23 04:08:38.792361041 +0000 UTC m=+801.253504713"
Jul 23 04:08:49.781408 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:08:49.781335    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b76d45f8-lz4rv"
Jul 23 04:09:20.257421 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:20.257375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:20.356564 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:20.356522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:20.457516 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:20.457480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:26.643784 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:26.643749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:26.749750 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:26.749707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:26.858930 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:26.858885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:27.085563 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:27.085521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:33.346730 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:33.346691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:09:33.446043 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:09:33.446004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:11:11.271986 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:11:11.271950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:12:46.080103 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:12:46.080067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:12:46.155308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:12:46.155279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:13:20.882420 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:20.882382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:13:26.179389 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:26.179354    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:13:26.179919 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:26.179542    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" podUID="561d5998-2680-4219-9468-214ae159729e" containerName="manager" containerID="cri-o://061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01" gracePeriod=10
Jul 23 04:13:28.834112 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:28.834083    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:13:28.920190 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:28.920126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume\") pod \"561d5998-2680-4219-9468-214ae159729e\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") "
Jul 23 04:13:28.920190 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:28.920162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9mpj\" (UniqueName: \"kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj\") pod \"561d5998-2680-4219-9468-214ae159729e\" (UID: \"561d5998-2680-4219-9468-214ae159729e\") "
Jul 23 04:13:28.920528 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:28.920507    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "561d5998-2680-4219-9468-214ae159729e" (UID: "561d5998-2680-4219-9468-214ae159729e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:13:28.922227 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:28.922205    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj" (OuterVolumeSpecName: "kube-api-access-k9mpj") pod "561d5998-2680-4219-9468-214ae159729e" (UID: "561d5998-2680-4219-9468-214ae159729e"). InnerVolumeSpecName "kube-api-access-k9mpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:13:29.021014 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.020990    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/561d5998-2680-4219-9468-214ae159729e-extensions-socket-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:13:29.021014 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.021012    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9mpj\" (UniqueName: \"kubernetes.io/projected/561d5998-2680-4219-9468-214ae159729e-kube-api-access-k9mpj\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:13:29.192106 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.192021    2582 generic.go:356] "Generic (PLEG): container finished" podID="561d5998-2680-4219-9468-214ae159729e" containerID="061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01" exitCode=0
Jul 23 04:13:29.192106 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.192071    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"
Jul 23 04:13:29.192299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.192105    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" event={"ID":"561d5998-2680-4219-9468-214ae159729e","Type":"ContainerDied","Data":"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"}
Jul 23 04:13:29.192299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.192151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk" event={"ID":"561d5998-2680-4219-9468-214ae159729e","Type":"ContainerDied","Data":"3bbaebc9919541c7d718efc97b40bb6fe3382ca5ff40c98cc0ff7f5c62a81b3f"}
Jul 23 04:13:29.192299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.192173    2582 scope.go:117] "RemoveContainer" containerID="061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"
Jul 23 04:13:29.201567 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.201552    2582 scope.go:117] "RemoveContainer" containerID="061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"
Jul 23 04:13:29.201846 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:13:29.201791    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01\": container with ID starting with 061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01 not found: ID does not exist" containerID="061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"
Jul 23 04:13:29.201910 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.201853    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01"} err="failed to get container status \"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01\": rpc error: code = NotFound desc = could not find container \"061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01\": container with ID starting with 061961c5bc2829d9b7d19da29d5c8c3ac26fe32a7c3c7585640d01e519133c01 not found: ID does not exist"
Jul 23 04:13:29.220006 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.219981    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:13:29.227945 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:29.227922    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vfsfk"]
Jul 23 04:13:30.053389 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:13:30.053355    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561d5998-2680-4219-9468-214ae159729e" path="/var/lib/kubelet/pods/561d5998-2680-4219-9468-214ae159729e/volumes"
Jul 23 04:14:27.320520 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.320489    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"]
Jul 23 04:14:27.320968 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.320913    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561d5998-2680-4219-9468-214ae159729e" containerName="manager"
Jul 23 04:14:27.320968 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.320927    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="561d5998-2680-4219-9468-214ae159729e" containerName="manager"
Jul 23 04:14:27.321045 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.321005    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="561d5998-2680-4219-9468-214ae159729e" containerName="manager"
Jul 23 04:14:27.325272 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.325256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.327971 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.327940    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qrd4w\""
Jul 23 04:14:27.335899 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.335871    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"]
Jul 23 04:14:27.403629 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.403606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/103d5bc4-827e-43ed-be5c-22d1d4764ee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.403740 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.403639    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvr9j\" (UniqueName: \"kubernetes.io/projected/103d5bc4-827e-43ed-be5c-22d1d4764ee3-kube-api-access-mvr9j\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.504958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.504935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/103d5bc4-827e-43ed-be5c-22d1d4764ee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.505060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.504964    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvr9j\" (UniqueName: \"kubernetes.io/projected/103d5bc4-827e-43ed-be5c-22d1d4764ee3-kube-api-access-mvr9j\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.505273 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.505256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/103d5bc4-827e-43ed-be5c-22d1d4764ee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.513327 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.513307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvr9j\" (UniqueName: \"kubernetes.io/projected/103d5bc4-827e-43ed-be5c-22d1d4764ee3-kube-api-access-mvr9j\") pod \"kuadrant-operator-controller-manager-86878c769f-lcgsk\" (UID: \"103d5bc4-827e-43ed-be5c-22d1d4764ee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.636585 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.636522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:27.787537 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:27.787515    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"]
Jul 23 04:14:27.789240 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:14:27.789215    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103d5bc4_827e_43ed_be5c_22d1d4764ee3.slice/crio-3f56408164dc6ff3a4b1ad4f5adb0139c5711b60d9ee4bb30bc7e7e9d58b169d WatchSource:0}: Error finding container 3f56408164dc6ff3a4b1ad4f5adb0139c5711b60d9ee4bb30bc7e7e9d58b169d: Status 404 returned error can't find the container with id 3f56408164dc6ff3a4b1ad4f5adb0139c5711b60d9ee4bb30bc7e7e9d58b169d
Jul 23 04:14:28.474049 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:28.474018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk" event={"ID":"103d5bc4-827e-43ed-be5c-22d1d4764ee3","Type":"ContainerStarted","Data":"f074c3975e3312faffe9363005672e64fd08ba7fc32dc5806aa31c082cc91225"}
Jul 23 04:14:28.474438 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:28.474066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk" event={"ID":"103d5bc4-827e-43ed-be5c-22d1d4764ee3","Type":"ContainerStarted","Data":"3f56408164dc6ff3a4b1ad4f5adb0139c5711b60d9ee4bb30bc7e7e9d58b169d"}
Jul 23 04:14:28.474438 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:28.474105    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:14:28.502546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:28.502492    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk" podStartSLOduration=1.5024774779999999 podStartE2EDuration="1.502477478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:14:28.494152766 +0000 UTC m=+1150.955296463" watchObservedRunningTime="2026-07-23 04:14:28.502477478 +0000 UTC m=+1150.963621217"
Jul 23 04:14:39.482051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:14:39.482018    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lcgsk"
Jul 23 04:16:21.633604 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:16:21.633573    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:17:16.798223 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:17:16.798190    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:17:23.906269 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:17:23.906240    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:17:28.548541 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:17:28.548511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:17:44.647223 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:17:44.647147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:17:45.412657 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:17:45.412625    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:20:42.388957 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:20:42.388884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:20:52.796879 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:20:52.796841    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:12.250694 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:12.250662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:12.302721 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:12.302694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:16.347861 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:16.347828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:19.320838 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:19.320779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:19.417286 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:19.417257    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:19.786359 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:19.786310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:25.476878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:25.476847    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:26.645347 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:26.645299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:28.271490 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.271460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:28.362056 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.362022    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:21:28.365535 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.365518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.368928 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.368902    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0ba06924-serving-cert\""
Jul 23 04:21:28.369626 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.369592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:28.376616 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.376592    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:21:28.453530 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.453506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:28.505597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.505572    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8pk\" (UniqueName: \"kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.505720 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.505639    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.505720 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.505660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.568586 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.568513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:28.606907 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.606881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.607015 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.606910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.607015 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.606946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8pk\" (UniqueName: \"kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.607543 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.607517    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.609356 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.609336    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.621182 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.621162    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8pk\" (UniqueName: \"kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk\") pod \"maas-api-e2e-ait-0ba06924-59995d7646-4pjb8\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.677871 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.677841    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:28.957481 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:28.957393    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:29.027772 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:29.027724    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:21:29.030513 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:21:29.030477    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937dab24_ea16_4166_938c_7d90164e479f.slice/crio-45f287eac4565f00cfafe72d1dfbf370c1f749ad6040c1b1846b3cc75acfa6d7 WatchSource:0}: Error finding container 45f287eac4565f00cfafe72d1dfbf370c1f749ad6040c1b1846b3cc75acfa6d7: Status 404 returned error can't find the container with id 45f287eac4565f00cfafe72d1dfbf370c1f749ad6040c1b1846b3cc75acfa6d7
Jul 23 04:21:29.032315 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:29.032297    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:21:29.047437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:29.047404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:21:29.488439 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:29.488409    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" event={"ID":"937dab24-ea16-4166-938c-7d90164e479f","Type":"ContainerStarted","Data":"45f287eac4565f00cfafe72d1dfbf370c1f749ad6040c1b1846b3cc75acfa6d7"}
Jul 23 04:21:30.494857 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:30.494798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" event={"ID":"937dab24-ea16-4166-938c-7d90164e479f","Type":"ContainerStarted","Data":"96438254e1a45c0d8dc352f5069a2bb5ae3f550da8e93c15b41c50721b6ebfaf"}
Jul 23 04:21:30.495230 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:30.494945    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:21:30.513284 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:30.513245    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" podStartSLOduration=1.900020682 podStartE2EDuration="2.513231679s" totalImagesPullingTime="613.210997ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:21:30.511500447 +0000 UTC m=+1572.972644120" watchObservedRunningTime="2026-07-23 04:21:30.513231679 +0000 UTC m=+1572.974375401"
Jul 23 04:21:36.505640 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:21:36.505614    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:22:04.431283 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.431249    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:22:04.431755 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.431478    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" podUID="937dab24-ea16-4166-938c-7d90164e479f" containerName="maas-api" containerID="cri-o://96438254e1a45c0d8dc352f5069a2bb5ae3f550da8e93c15b41c50721b6ebfaf" gracePeriod=30
Jul 23 04:22:04.457021 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.456995    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:04.658728 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.658698    2582 generic.go:356] "Generic (PLEG): container finished" podID="937dab24-ea16-4166-938c-7d90164e479f" containerID="96438254e1a45c0d8dc352f5069a2bb5ae3f550da8e93c15b41c50721b6ebfaf" exitCode=0
Jul 23 04:22:04.658897 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.658771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" event={"ID":"937dab24-ea16-4166-938c-7d90164e479f","Type":"ContainerDied","Data":"96438254e1a45c0d8dc352f5069a2bb5ae3f550da8e93c15b41c50721b6ebfaf"}
Jul 23 04:22:04.685379 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.681824    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:22:04.707506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.707480    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj8pk\" (UniqueName: \"kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk\") pod \"937dab24-ea16-4166-938c-7d90164e479f\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") "
Jul 23 04:22:04.707623 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.707516    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca\") pod \"937dab24-ea16-4166-938c-7d90164e479f\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") "
Jul 23 04:22:04.707623 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.707558    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls\") pod \"937dab24-ea16-4166-938c-7d90164e479f\" (UID: \"937dab24-ea16-4166-938c-7d90164e479f\") "
Jul 23 04:22:04.707931 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.707905    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "937dab24-ea16-4166-938c-7d90164e479f" (UID: "937dab24-ea16-4166-938c-7d90164e479f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:22:04.709737 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.709711    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "937dab24-ea16-4166-938c-7d90164e479f" (UID: "937dab24-ea16-4166-938c-7d90164e479f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:22:04.709955 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.709938    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk" (OuterVolumeSpecName: "kube-api-access-nj8pk") pod "937dab24-ea16-4166-938c-7d90164e479f" (UID: "937dab24-ea16-4166-938c-7d90164e479f"). InnerVolumeSpecName "kube-api-access-nj8pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:22:04.808326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.808297    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj8pk\" (UniqueName: \"kubernetes.io/projected/937dab24-ea16-4166-938c-7d90164e479f-kube-api-access-nj8pk\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:04.808326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.808322    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/937dab24-ea16-4166-938c-7d90164e479f-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:04.808469 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:04.808333    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/937dab24-ea16-4166-938c-7d90164e479f-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:05.664749 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:05.664711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8" event={"ID":"937dab24-ea16-4166-938c-7d90164e479f","Type":"ContainerDied","Data":"45f287eac4565f00cfafe72d1dfbf370c1f749ad6040c1b1846b3cc75acfa6d7"}
Jul 23 04:22:05.665240 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:05.664780    2582 scope.go:117] "RemoveContainer" containerID="96438254e1a45c0d8dc352f5069a2bb5ae3f550da8e93c15b41c50721b6ebfaf"
Jul 23 04:22:05.665240 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:05.664732    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"
Jul 23 04:22:05.686622 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:05.686597    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:22:05.690761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:05.690740    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ba06924-59995d7646-4pjb8"]
Jul 23 04:22:06.053153 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:06.053122    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937dab24-ea16-4166-938c-7d90164e479f" path="/var/lib/kubelet/pods/937dab24-ea16-4166-938c-7d90164e479f/volumes"
Jul 23 04:22:08.942504 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:08.942472    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:09.229931 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:09.229903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:09.392756 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:09.392722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:09.656606 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:09.656536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:09.801737 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:09.801707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:10.007260 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:10.007230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:10.308218 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:10.308081    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:10.560100 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:10.560028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:12.635520 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.635485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:12.709316 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.709288    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:12.709666 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.709654    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937dab24-ea16-4166-938c-7d90164e479f" containerName="maas-api"
Jul 23 04:22:12.709717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.709667    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="937dab24-ea16-4166-938c-7d90164e479f" containerName="maas-api"
Jul 23 04:22:12.709756 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.709729    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="937dab24-ea16-4166-938c-7d90164e479f" containerName="maas-api"
Jul 23 04:22:12.714047 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.714026    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.716518 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.716498    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-8238e07c-serving-cert\""
Jul 23 04:22:12.726078 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.726056    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:12.743343 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.743311    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:12.775776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.775755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.775886 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.775782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.775886 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.775839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bjwn\" (UniqueName: \"kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.876392 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.876361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.876392 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.876395    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.876636 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.876437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bjwn\" (UniqueName: \"kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.876636 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:22:12.876534    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-serving-cert: secret "maas-api-e2e-migrate-8238e07c-serving-cert" not found
Jul 23 04:22:12.876636 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:22:12.876610    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls podName:e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60 nodeName:}" failed. No retries permitted until 2026-07-23 04:22:13.376589589 +0000 UTC m=+1615.837733241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls") pod "maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" (UID: "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60") : secret "maas-api-e2e-migrate-8238e07c-serving-cert" not found
Jul 23 04:22:12.877138 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.877119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:12.884946 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:12.884927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bjwn\" (UniqueName: \"kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:13.091244 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.091203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:13.229922 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.229886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:13.330865 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.330832    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:13.380442 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.380366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:13.382930 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.382903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") pod \"maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:13.625350 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.625314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:13.764569 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:13.764544    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:13.766292 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:22:13.766261    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fac35d_b4a3_4bfc_a9f7_ecbce1fe0e60.slice/crio-f9196682cbdc2238995276e5c31687d23f48fb89166bdbcf0d000784c6026650 WatchSource:0}: Error finding container f9196682cbdc2238995276e5c31687d23f48fb89166bdbcf0d000784c6026650: Status 404 returned error can't find the container with id f9196682cbdc2238995276e5c31687d23f48fb89166bdbcf0d000784c6026650
Jul 23 04:22:14.711683 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:14.711645    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" event={"ID":"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60","Type":"ContainerStarted","Data":"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"}
Jul 23 04:22:14.711885 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:14.711699    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" event={"ID":"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60","Type":"ContainerStarted","Data":"f9196682cbdc2238995276e5c31687d23f48fb89166bdbcf0d000784c6026650"}
Jul 23 04:22:14.711885 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:14.711774    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:14.730389 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:14.730349    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" podStartSLOduration=2.32903939 podStartE2EDuration="2.730334546s" totalImagesPullingTime="401.295156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:22:14.728499154 +0000 UTC m=+1617.189642852" watchObservedRunningTime="2026-07-23 04:22:14.730334546 +0000 UTC m=+1617.191478218"
Jul 23 04:22:20.722521 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:20.722490    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:48.078701 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.078668    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:48.079907 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.079770    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" podUID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" containerName="maas-api" containerID="cri-o://d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff" gracePeriod=30
Jul 23 04:22:48.112794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.112762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:48.330864 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.330793    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:48.476771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.476745    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca\") pod \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") "
Jul 23 04:22:48.476950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.476853    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") pod \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") "
Jul 23 04:22:48.476950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.476894    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bjwn\" (UniqueName: \"kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn\") pod \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\" (UID: \"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60\") "
Jul 23 04:22:48.477108 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.477085    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" (UID: "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:22:48.478903 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.478882    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" (UID: "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:22:48.479000 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.478986    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn" (OuterVolumeSpecName: "kube-api-access-5bjwn") pod "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" (UID: "e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60"). InnerVolumeSpecName "kube-api-access-5bjwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:22:48.578166 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.578140    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:48.578166 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.578164    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bjwn\" (UniqueName: \"kubernetes.io/projected/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-kube-api-access-5bjwn\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:48.578309 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.578174    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:22:48.875884 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.875851    2582 generic.go:356] "Generic (PLEG): container finished" podID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" containerID="d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff" exitCode=0
Jul 23 04:22:48.876030 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.875905    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"
Jul 23 04:22:48.876030 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.875941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" event={"ID":"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60","Type":"ContainerDied","Data":"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"}
Jul 23 04:22:48.876161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.876029    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk" event={"ID":"e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60","Type":"ContainerDied","Data":"f9196682cbdc2238995276e5c31687d23f48fb89166bdbcf0d000784c6026650"}
Jul 23 04:22:48.876161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.876061    2582 scope.go:117] "RemoveContainer" containerID="d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"
Jul 23 04:22:48.885722 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.885704    2582 scope.go:117] "RemoveContainer" containerID="d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"
Jul 23 04:22:48.885997 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:22:48.885977    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff\": container with ID starting with d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff not found: ID does not exist" containerID="d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"
Jul 23 04:22:48.886059 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.886004    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff"} err="failed to get container status \"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff\": rpc error: code = NotFound desc = could not find container \"d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff\": container with ID starting with d82898aad5050f81e8fdfe4cfadd1821f7223f8bd2189dcf87a878636da516ff not found: ID does not exist"
Jul 23 04:22:48.900727 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.900698    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:48.902517 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:48.902493    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8238e07c-79966dcff9-6ckpk"]
Jul 23 04:22:50.054286 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:50.054253    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" path="/var/lib/kubelet/pods/e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60/volumes"
Jul 23 04:22:52.736787 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:52.736750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:52.877771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:52.877730    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:53.109592 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:53.109509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:53.359796 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:53.359721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:53.698170 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:53.698080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:53.867014 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:53.866985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:55.823573 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.823536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:55.908923 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.908890    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:22:55.909334 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.909318    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" containerName="maas-api"
Jul 23 04:22:55.909396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.909336    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" containerName="maas-api"
Jul 23 04:22:55.909434 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.909395    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6fac35d-b4a3-4bfc-a9f7-ecbce1fe0e60" containerName="maas-api"
Jul 23 04:22:55.912527 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.912510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:55.916499 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.916472    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:55.918415 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.918396    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5116e3b8-serving-cert\""
Jul 23 04:22:55.931523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.931499    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:22:55.936856 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.936832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:55.936974 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.936872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbdj9\" (UniqueName: \"kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:55.937040 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:55.936984    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.038323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.038291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.038323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.038326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbdj9\" (UniqueName: \"kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.038501 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.038349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.038501 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:22:56.038464    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-serving-cert: secret "maas-api-e2e-ait-5116e3b8-serving-cert" not found
Jul 23 04:22:56.038587 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:22:56.038524    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls podName:18787099-3c21-4a9c-8ed0-1f632ef26676 nodeName:}" failed. No retries permitted until 2026-07-23 04:22:56.538504531 +0000 UTC m=+1658.999648181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls") pod "maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" (UID: "18787099-3c21-4a9c-8ed0-1f632ef26676") : secret "maas-api-e2e-ait-5116e3b8-serving-cert" not found
Jul 23 04:22:56.039076 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.039052    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.044262 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.044240    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:56.077839 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.077765    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbdj9\" (UniqueName: \"kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.201466 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.201432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:56.543536 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.543504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.545894 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.545858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") pod \"maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.639073 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.639038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:56.742062 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.742035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:22:56.822577 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.822508    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:56.961414 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:56.961390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:22:56.963627 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:22:56.963603    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18787099_3c21_4a9c_8ed0_1f632ef26676.slice/crio-bd9fc8ad14143f047bd48dc452f678a087f7f4dd956fc2d41c7792e78e66ff82 WatchSource:0}: Error finding container bd9fc8ad14143f047bd48dc452f678a087f7f4dd956fc2d41c7792e78e66ff82: Status 404 returned error can't find the container with id bd9fc8ad14143f047bd48dc452f678a087f7f4dd956fc2d41c7792e78e66ff82
Jul 23 04:22:57.924546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:57.924511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" event={"ID":"18787099-3c21-4a9c-8ed0-1f632ef26676","Type":"ContainerStarted","Data":"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"}
Jul 23 04:22:57.924701 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:57.924570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" event={"ID":"18787099-3c21-4a9c-8ed0-1f632ef26676","Type":"ContainerStarted","Data":"bd9fc8ad14143f047bd48dc452f678a087f7f4dd956fc2d41c7792e78e66ff82"}
Jul 23 04:22:57.924701 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:57.924654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:22:57.943916 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:22:57.943871    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" podStartSLOduration=2.467248875 podStartE2EDuration="2.943853852s" totalImagesPullingTime="476.604977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:22:57.942856748 +0000 UTC m=+1660.404000447" watchObservedRunningTime="2026-07-23 04:22:57.943853852 +0000 UTC m=+1660.404997525"
Jul 23 04:23:03.935350 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:03.935321    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:23:32.028450 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.028415    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:23:32.028953 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.028617    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" podUID="18787099-3c21-4a9c-8ed0-1f632ef26676" containerName="maas-api" containerID="cri-o://201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e" gracePeriod=30
Jul 23 04:23:32.062633 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.062609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:32.278814 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.278750    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:23:32.446391 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.446361    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") pod \"18787099-3c21-4a9c-8ed0-1f632ef26676\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") "
Jul 23 04:23:32.446537 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.446420    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbdj9\" (UniqueName: \"kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9\") pod \"18787099-3c21-4a9c-8ed0-1f632ef26676\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") "
Jul 23 04:23:32.446537 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.446449    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca\") pod \"18787099-3c21-4a9c-8ed0-1f632ef26676\" (UID: \"18787099-3c21-4a9c-8ed0-1f632ef26676\") "
Jul 23 04:23:32.446921 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.446885    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18787099-3c21-4a9c-8ed0-1f632ef26676" (UID: "18787099-3c21-4a9c-8ed0-1f632ef26676"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:23:32.448429 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.448396    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9" (OuterVolumeSpecName: "kube-api-access-bbdj9") pod "18787099-3c21-4a9c-8ed0-1f632ef26676" (UID: "18787099-3c21-4a9c-8ed0-1f632ef26676"). InnerVolumeSpecName "kube-api-access-bbdj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:23:32.448544 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.448428    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18787099-3c21-4a9c-8ed0-1f632ef26676" (UID: "18787099-3c21-4a9c-8ed0-1f632ef26676"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:23:32.547276 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.547221    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18787099-3c21-4a9c-8ed0-1f632ef26676-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:23:32.547276 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.547243    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18787099-3c21-4a9c-8ed0-1f632ef26676-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:23:32.547276 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:32.547252    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbdj9\" (UniqueName: \"kubernetes.io/projected/18787099-3c21-4a9c-8ed0-1f632ef26676-kube-api-access-bbdj9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:23:33.101140 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.101108    2582 generic.go:356] "Generic (PLEG): container finished" podID="18787099-3c21-4a9c-8ed0-1f632ef26676" containerID="201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e" exitCode=0
Jul 23 04:23:33.101638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.101160    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"
Jul 23 04:23:33.101638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.101195    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" event={"ID":"18787099-3c21-4a9c-8ed0-1f632ef26676","Type":"ContainerDied","Data":"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"}
Jul 23 04:23:33.101638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.101241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95" event={"ID":"18787099-3c21-4a9c-8ed0-1f632ef26676","Type":"ContainerDied","Data":"bd9fc8ad14143f047bd48dc452f678a087f7f4dd956fc2d41c7792e78e66ff82"}
Jul 23 04:23:33.101638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.101263    2582 scope.go:117] "RemoveContainer" containerID="201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"
Jul 23 04:23:33.110356 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.110341    2582 scope.go:117] "RemoveContainer" containerID="201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"
Jul 23 04:23:33.110594 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:23:33.110573    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e\": container with ID starting with 201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e not found: ID does not exist" containerID="201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"
Jul 23 04:23:33.110655 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.110607    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e"} err="failed to get container status \"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e\": rpc error: code = NotFound desc = could not find container \"201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e\": container with ID starting with 201b89005f9a9b563103df1d6a6499035de2616675406102748549ab1d68779e not found: ID does not exist"
Jul 23 04:23:33.124117 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.124085    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:23:33.128446 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:33.128425    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5116e3b8-55cb57c8fc-gpz95"]
Jul 23 04:23:34.052382 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:34.052344    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18787099-3c21-4a9c-8ed0-1f632ef26676" path="/var/lib/kubelet/pods/18787099-3c21-4a9c-8ed0-1f632ef26676/volumes"
Jul 23 04:23:36.674404 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:36.674374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:37.900572 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:37.900535    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:38.127516 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.127488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:38.352198 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.352163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:38.649297 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.649213    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:23:38.649691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.649672    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18787099-3c21-4a9c-8ed0-1f632ef26676" containerName="maas-api"
Jul 23 04:23:38.649778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.649694    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18787099-3c21-4a9c-8ed0-1f632ef26676" containerName="maas-api"
Jul 23 04:23:38.649778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.649775    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18787099-3c21-4a9c-8ed0-1f632ef26676" containerName="maas-api"
Jul 23 04:23:38.653098 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.653076    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.655604 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.655584    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d9dfcbba-openshift-default-dockercfg-fpx7c\""
Jul 23 04:23:38.667026 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.667004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:23:38.676958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.676936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:38.784921 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.784886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:38.804142 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804268 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804263    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804470 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwr2q\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804470 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.804470 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.804393    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.904893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.904893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.904893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904875    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.904893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.904985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwr2q\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905061    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905262    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905339    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.905945 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.905588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.907533 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.907509    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.907715 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.907697    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.916258 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.916238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.916567 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.916551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwr2q\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q\") pod \"e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") " pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:38.965383 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:38.965355    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:39.100492 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:39.100468    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:23:39.102017 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:23:39.101988    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17beafe9_d57c_4db1_9ded_d6b5a5c758d9.slice/crio-1d4b73d6d4dd00dfef3c86b12e1ec96c0c88600be9e80f9a2ef30bcc2be6eb02 WatchSource:0}: Error finding container 1d4b73d6d4dd00dfef3c86b12e1ec96c0c88600be9e80f9a2ef30bcc2be6eb02: Status 404 returned error can't find the container with id 1d4b73d6d4dd00dfef3c86b12e1ec96c0c88600be9e80f9a2ef30bcc2be6eb02
Jul 23 04:23:39.104125 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:39.104094    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:39.104193 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:39.104159    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:39.104238 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:39.104189    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:39.130632 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:39.130607    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerStarted","Data":"1d4b73d6d4dd00dfef3c86b12e1ec96c0c88600be9e80f9a2ef30bcc2be6eb02"}
Jul 23 04:23:40.137342 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:40.137300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerStarted","Data":"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"}
Jul 23 04:23:40.164205 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:40.164157    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podStartSLOduration=2.164141737 podStartE2EDuration="2.164141737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:23:40.163007897 +0000 UTC m=+1702.624196050" watchObservedRunningTime="2026-07-23 04:23:40.164141737 +0000 UTC m=+1702.625285409"
Jul 23 04:23:40.965995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:40.965960    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:40.970701 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:40.970678    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:41.143753 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.143712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:41.144918 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.144898    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:41.507022 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.506986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:41.609547 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.609496    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:41.696477 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.696448    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:23:41.700024 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.700007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.702615 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.702592    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d9dfcbba-serving-cert\""
Jul 23 04:23:41.713457 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.713433    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:23:41.723314 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.723292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:41.832123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.832037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.832123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.832096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.832347 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.832171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw9rx\" (UniqueName: \"kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.933639 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.933605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.933785 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.933670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw9rx\" (UniqueName: \"kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.933785 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.933773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.934004 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:23:41.933984    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-serving-cert: secret "maas-api-e2e-derive-d9dfcbba-serving-cert" not found
Jul 23 04:23:41.934068 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:23:41.934058    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls podName:23c4726e-0933-4e24-8a35-110ac4a99cc3 nodeName:}" failed. No retries permitted until 2026-07-23 04:23:42.434036956 +0000 UTC m=+1704.895180607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls") pod "maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" (UID: "23c4726e-0933-4e24-8a35-110ac4a99cc3") : secret "maas-api-e2e-derive-d9dfcbba-serving-cert" not found
Jul 23 04:23:41.934334 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.934318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:41.946582 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:41.946552    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw9rx\" (UniqueName: \"kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:42.176326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.176246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:42.403895 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.403851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:42.439664 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.439587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:42.441876 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.441848    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") pod \"maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:42.505436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.505408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:23:42.610976 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.610947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:42.752040 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:42.752018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:23:42.754236 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:23:42.754211    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c4726e_0933_4e24_8a35_110ac4a99cc3.slice/crio-4c4965ce12a7b384e98014f3ccf4494811f19f5ffbd1ce6306aa16597b45076e WatchSource:0}: Error finding container 4c4965ce12a7b384e98014f3ccf4494811f19f5ffbd1ce6306aa16597b45076e: Status 404 returned error can't find the container with id 4c4965ce12a7b384e98014f3ccf4494811f19f5ffbd1ce6306aa16597b45076e
Jul 23 04:23:43.153416 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:43.153393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" event={"ID":"23c4726e-0933-4e24-8a35-110ac4a99cc3","Type":"ContainerStarted","Data":"4c4965ce12a7b384e98014f3ccf4494811f19f5ffbd1ce6306aa16597b45076e"}
Jul 23 04:23:44.160066 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:44.160033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" event={"ID":"23c4726e-0933-4e24-8a35-110ac4a99cc3","Type":"ContainerStarted","Data":"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"}
Jul 23 04:23:44.160413 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:44.160144    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:44.180910 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:44.180863    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" podStartSLOduration=2.833075178 podStartE2EDuration="3.180850387s" totalImagesPullingTime="347.775209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:23:44.178758908 +0000 UTC m=+1706.639902601" watchObservedRunningTime="2026-07-23 04:23:44.180850387 +0000 UTC m=+1706.641994058"
Jul 23 04:23:50.170566 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:50.170536    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:23:55.218691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.218661    2582 generic.go:356] "Generic (PLEG): container finished" podID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerID="d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704" exitCode=0
Jul 23 04:23:55.219042 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.218737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerDied","Data":"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"}
Jul 23 04:23:55.219175 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.219158    2582 scope.go:117] "RemoveContainer" containerID="d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"
Jul 23 04:23:55.220142 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.220117    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:55.220202 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.220189    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:55.220242 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:55.220225    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:23:56.143791 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:56.143733    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:56.225082 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:56.225041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerStarted","Data":"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"}
Jul 23 04:23:56.966437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:56.966405    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:23:56.967667 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:56.967636    2582 patch_prober.go:28] interesting pod/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 04:23:56.967794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:56.967687    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 04:23:57.965856 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:57.965795    2582 patch_prober.go:28] interesting pod/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 04:23:57.966215 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:57.965885    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 04:23:58.965854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:58.965797    2582 patch_prober.go:28] interesting pod/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 04:23:58.966036 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:58.965884    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 04:23:59.966156 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:59.966116    2582 patch_prober.go:28] interesting pod/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 04:23:59.966542 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:23:59.966175    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 04:24:00.267898 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:00.267860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:00.367861 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:00.367825    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:00.974826 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:00.969576    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:24:00.974826 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:00.970837    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:24:00.976289 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:00.976261    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:24:04.145044 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:04.145007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:04.245776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:04.245747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:16.905940 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:16.905909    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:24:16.906389 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:16.906163    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" podUID="23c4726e-0933-4e24-8a35-110ac4a99cc3" containerName="maas-api" containerID="cri-o://ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656" gracePeriod=30
Jul 23 04:24:16.937479 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:16.937453    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:17.182467 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.182446    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:24:17.241845 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.241795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca\") pod \"23c4726e-0933-4e24-8a35-110ac4a99cc3\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") "
Jul 23 04:24:17.241995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.241880    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw9rx\" (UniqueName: \"kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx\") pod \"23c4726e-0933-4e24-8a35-110ac4a99cc3\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") "
Jul 23 04:24:17.241995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.241941    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") pod \"23c4726e-0933-4e24-8a35-110ac4a99cc3\" (UID: \"23c4726e-0933-4e24-8a35-110ac4a99cc3\") "
Jul 23 04:24:17.242149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.242126    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23c4726e-0933-4e24-8a35-110ac4a99cc3" (UID: "23c4726e-0933-4e24-8a35-110ac4a99cc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:24:17.243971 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.243943    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23c4726e-0933-4e24-8a35-110ac4a99cc3" (UID: "23c4726e-0933-4e24-8a35-110ac4a99cc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:24:17.244076 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.243989    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx" (OuterVolumeSpecName: "kube-api-access-hw9rx") pod "23c4726e-0933-4e24-8a35-110ac4a99cc3" (UID: "23c4726e-0933-4e24-8a35-110ac4a99cc3"). InnerVolumeSpecName "kube-api-access-hw9rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:24:17.335922 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.335889    2582 generic.go:356] "Generic (PLEG): container finished" podID="23c4726e-0933-4e24-8a35-110ac4a99cc3" containerID="ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656" exitCode=0
Jul 23 04:24:17.336062 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.335951    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"
Jul 23 04:24:17.336062 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.335974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" event={"ID":"23c4726e-0933-4e24-8a35-110ac4a99cc3","Type":"ContainerDied","Data":"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"}
Jul 23 04:24:17.336062 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.336031    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7" event={"ID":"23c4726e-0933-4e24-8a35-110ac4a99cc3","Type":"ContainerDied","Data":"4c4965ce12a7b384e98014f3ccf4494811f19f5ffbd1ce6306aa16597b45076e"}
Jul 23 04:24:17.336062 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.336056    2582 scope.go:117] "RemoveContainer" containerID="ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"
Jul 23 04:24:17.342636 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.342612    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c4726e-0933-4e24-8a35-110ac4a99cc3-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:17.342763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.342642    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c4726e-0933-4e24-8a35-110ac4a99cc3-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:17.342763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.342659    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw9rx\" (UniqueName: \"kubernetes.io/projected/23c4726e-0933-4e24-8a35-110ac4a99cc3-kube-api-access-hw9rx\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:17.345674 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.345658    2582 scope.go:117] "RemoveContainer" containerID="ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"
Jul 23 04:24:17.345992 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:24:17.345975    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656\": container with ID starting with ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656 not found: ID does not exist" containerID="ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"
Jul 23 04:24:17.346058 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.346001    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656"} err="failed to get container status \"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656\": rpc error: code = NotFound desc = could not find container \"ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656\": container with ID starting with ade32c5e40492388c67076f4ac8faab8cde1d40754e2bdd997432c91a7c2f656 not found: ID does not exist"
Jul 23 04:24:17.359216 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.359197    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:24:17.362753 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:17.362733    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d9dfcbba-6686cb886-lczv7"]
Jul 23 04:24:18.054863 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:18.054836    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c4726e-0933-4e24-8a35-110ac4a99cc3" path="/var/lib/kubelet/pods/23c4726e-0933-4e24-8a35-110ac4a99cc3/volumes"
Jul 23 04:24:21.476409 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:21.476359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:21.625992 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:21.625954    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:24:21.626177 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:21.626147    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy" containerID="cri-o://d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e" gracePeriod=30
Jul 23 04:24:21.634760 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:21.634740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:21.874622 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:21.874571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:22.350426 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:22.350349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:22.454362 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:22.454336    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:22.580078 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:22.580039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:23.984776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:23.984743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:26.874279 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:26.874258    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:24:27.030073 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030047    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwr2q\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030091    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030115    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030168    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030203    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030234    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030536 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030268    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030536 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030310    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030536 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030365    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data\") pod \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\" (UID: \"17beafe9-d57c-4db1-9ded-d6b5a5c758d9\") "
Jul 23 04:24:27.030536 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030369    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:24:27.030736 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:24:27.030736 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030597    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:24:27.030736 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030607    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:24:27.030927 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030874    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.030927 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030905    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.030927 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030920    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.031068 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.030935    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.031310 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.031289    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:24:27.032540 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.032514    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q" (OuterVolumeSpecName: "kube-api-access-zwr2q") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "kube-api-access-zwr2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:24:27.032780 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.032748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 04:24:27.032888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.032850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:24:27.034242 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.034216    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "17beafe9-d57c-4db1-9ded-d6b5a5c758d9" (UID: "17beafe9-d57c-4db1-9ded-d6b5a5c758d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:24:27.132249 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.132222    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwr2q\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-kube-api-access-zwr2q\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.132249 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.132248    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.132396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.132259    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.132396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.132269    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.132396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.132277    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17beafe9-d57c-4db1-9ded-d6b5a5c758d9-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:24:27.385045 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.384966    2582 generic.go:356] "Generic (PLEG): container finished" podID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerID="d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e" exitCode=0
Jul 23 04:24:27.385045 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.385018    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"
Jul 23 04:24:27.385045 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.385026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerDied","Data":"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"}
Jul 23 04:24:27.385311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.385068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk" event={"ID":"17beafe9-d57c-4db1-9ded-d6b5a5c758d9","Type":"ContainerDied","Data":"1d4b73d6d4dd00dfef3c86b12e1ec96c0c88600be9e80f9a2ef30bcc2be6eb02"}
Jul 23 04:24:27.385311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.385090    2582 scope.go:117] "RemoveContainer" containerID="d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"
Jul 23 04:24:27.394206 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.394182    2582 scope.go:117] "RemoveContainer" containerID="d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"
Jul 23 04:24:27.403881 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.403860    2582 scope.go:117] "RemoveContainer" containerID="d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"
Jul 23 04:24:27.404155 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:24:27.404133    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e\": container with ID starting with d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e not found: ID does not exist" containerID="d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"
Jul 23 04:24:27.404222 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.404166    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e"} err="failed to get container status \"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e\": rpc error: code = NotFound desc = could not find container \"d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e\": container with ID starting with d4608465f84a826b3a3fb83a6a98598cf30c476e6e68db0d7eb6b63d06a8c63e not found: ID does not exist"
Jul 23 04:24:27.404266 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.404226    2582 scope.go:117] "RemoveContainer" containerID="d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"
Jul 23 04:24:27.404519 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:24:27.404495    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704\": container with ID starting with d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704 not found: ID does not exist" containerID="d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"
Jul 23 04:24:27.404643 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.404526    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704"} err="failed to get container status \"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704\": rpc error: code = NotFound desc = could not find container \"d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704\": container with ID starting with d20b3c4b39635e1dcb31048102cfc3aacfdd032444ed015e6b58623f71d4b704 not found: ID does not exist"
Jul 23 04:24:27.406831 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.406781    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:24:27.410598 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:27.410575    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d9dfcbba-openshift-default-7cdb965f8c-5ptgk"]
Jul 23 04:24:28.052761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:28.052721    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" path="/var/lib/kubelet/pods/17beafe9-d57c-4db1-9ded-d6b5a5c758d9/volumes"
Jul 23 04:24:30.015643 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:30.015614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:30.123558 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:30.123521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:30.218864 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:30.218834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:30.318450 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:30.318377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:30.753051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:30.753021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:44.766995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:44.766959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:24:56.731402 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:24:56.731360    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:02.993220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:02.993178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:03.105603 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:03.105568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:03.201852 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:03.201788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:19.117518 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:19.117485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:24.868763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:24.868712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:24.930128 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:24.930097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:25.017334 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:25.017298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:25.115637 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:25.115607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:35.609687 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:35.609653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:36.396072 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:36.396040    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:42.812387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:42.812349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:42.920409 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:42.920375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:43.043691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:43.043659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:43.195508 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:43.195430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:43.957426 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:43.957394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:44.043935 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:44.043902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:45.757960 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:45.757923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:51.987557 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:51.987526    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:52.087436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:52.087407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:52.694676 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:52.694647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:58.528758 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:58.528728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:58.633496 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:58.633460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:58.759250 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:58.759223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:59.127191 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:59.127158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:25:59.348124 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:25:59.348092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:02.297756 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.297718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:02.406188 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406152    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:02.406645 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406622    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406645 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406647    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406659    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406668    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406685    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c4726e-0933-4e24-8a35-110ac4a99cc3" containerName="maas-api"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406694    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c4726e-0933-4e24-8a35-110ac4a99cc3" containerName="maas-api"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406790    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406828    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="17beafe9-d57c-4db1-9ded-d6b5a5c758d9" containerName="istio-proxy"
Jul 23 04:26:02.406836 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.406838    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="23c4726e-0933-4e24-8a35-110ac4a99cc3" containerName="maas-api"
Jul 23 04:26:02.409951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.409931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.412375 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.412352    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dc495b0d-serving-cert\""
Jul 23 04:26:02.412556 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.412373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:02.420726 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.420700    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:02.499011 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.498978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.499130 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.499018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4d2s\" (UniqueName: \"kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.499130 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.499096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.502858 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.502833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:02.599988 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.599904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.599988 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.599946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4d2s\" (UniqueName: \"kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.600210 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.600041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.600553 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.600529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.602387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.602362    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.628990 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.628963    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4d2s\" (UniqueName: \"kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s\") pod \"maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.722001 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.721974    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:02.867149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:02.867126    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:02.869171 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:26:02.869142    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82254d34_de69_41c8_8678_2cdbc930eec1.slice/crio-2cb3101ae7b3ffb0208c243d5f891fbe3985ae5c764f6744f532e9bb84737ac4 WatchSource:0}: Error finding container 2cb3101ae7b3ffb0208c243d5f891fbe3985ae5c764f6744f532e9bb84737ac4: Status 404 returned error can't find the container with id 2cb3101ae7b3ffb0208c243d5f891fbe3985ae5c764f6744f532e9bb84737ac4
Jul 23 04:26:03.152852 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.152752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:03.253407 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.253375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:03.849215 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.849181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" event={"ID":"82254d34-de69-41c8-8678-2cdbc930eec1","Type":"ContainerStarted","Data":"b034f8f806489b9b26b287cdd33c87756f954441bafb14b28b4e1cf318b25450"}
Jul 23 04:26:03.849600 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.849239    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:03.849600 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.849256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" event={"ID":"82254d34-de69-41c8-8678-2cdbc930eec1","Type":"ContainerStarted","Data":"2cb3101ae7b3ffb0208c243d5f891fbe3985ae5c764f6744f532e9bb84737ac4"}
Jul 23 04:26:03.869194 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:03.869145    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" podStartSLOduration=1.430811651 podStartE2EDuration="1.869131126s" totalImagesPullingTime="438.319475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:26:03.868188665 +0000 UTC m=+1846.329332338" watchObservedRunningTime="2026-07-23 04:26:03.869131126 +0000 UTC m=+1846.330274798"
Jul 23 04:26:06.783846 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:06.783795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:07.483440 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:07.483405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:07.582060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:07.582026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:07.928912 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:07.928834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:09.859253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:09.859226    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:14.009532 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:14.009501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:14.109476 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:14.109445    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:24.814941 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.814908    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:24.815472 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.815174    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" containerName="maas-api" containerID="cri-o://b034f8f806489b9b26b287cdd33c87756f954441bafb14b28b4e1cf318b25450" gracePeriod=30
Jul 23 04:26:24.840193 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.840169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:24.854482 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.854456    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": dial tcp 10.132.0.55:8443: connect: connection refused"
Jul 23 04:26:24.951820 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.951780    2582 generic.go:356] "Generic (PLEG): container finished" podID="82254d34-de69-41c8-8678-2cdbc930eec1" containerID="b034f8f806489b9b26b287cdd33c87756f954441bafb14b28b4e1cf318b25450" exitCode=0
Jul 23 04:26:24.951947 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:24.951836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" event={"ID":"82254d34-de69-41c8-8678-2cdbc930eec1","Type":"ContainerDied","Data":"b034f8f806489b9b26b287cdd33c87756f954441bafb14b28b4e1cf318b25450"}
Jul 23 04:26:25.061714 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.061692    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:25.190890 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.190821    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca\") pod \"82254d34-de69-41c8-8678-2cdbc930eec1\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") "
Jul 23 04:26:25.190890 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.190861    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4d2s\" (UniqueName: \"kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s\") pod \"82254d34-de69-41c8-8678-2cdbc930eec1\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") "
Jul 23 04:26:25.191092 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.190909    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls\") pod \"82254d34-de69-41c8-8678-2cdbc930eec1\" (UID: \"82254d34-de69-41c8-8678-2cdbc930eec1\") "
Jul 23 04:26:25.191155 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.191097    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82254d34-de69-41c8-8678-2cdbc930eec1" (UID: "82254d34-de69-41c8-8678-2cdbc930eec1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:26:25.191212 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.191200    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82254d34-de69-41c8-8678-2cdbc930eec1-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:26:25.192886 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.192857    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82254d34-de69-41c8-8678-2cdbc930eec1" (UID: "82254d34-de69-41c8-8678-2cdbc930eec1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:26:25.192992 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.192886    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s" (OuterVolumeSpecName: "kube-api-access-m4d2s") pod "82254d34-de69-41c8-8678-2cdbc930eec1" (UID: "82254d34-de69-41c8-8678-2cdbc930eec1"). InnerVolumeSpecName "kube-api-access-m4d2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:26:25.292385 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.292365    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82254d34-de69-41c8-8678-2cdbc930eec1-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:26:25.292385 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.292385    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4d2s\" (UniqueName: \"kubernetes.io/projected/82254d34-de69-41c8-8678-2cdbc930eec1-kube-api-access-m4d2s\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:26:25.957615 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.957579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z" event={"ID":"82254d34-de69-41c8-8678-2cdbc930eec1","Type":"ContainerDied","Data":"2cb3101ae7b3ffb0208c243d5f891fbe3985ae5c764f6744f532e9bb84737ac4"}
Jul 23 04:26:25.957615 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.957598    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"
Jul 23 04:26:25.958167 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.957686    2582 scope.go:117] "RemoveContainer" containerID="b034f8f806489b9b26b287cdd33c87756f954441bafb14b28b4e1cf318b25450"
Jul 23 04:26:25.984303 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.984278    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:25.988488 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:25.988466    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dc495b0d-598d9d679b-rz99z"]
Jul 23 04:26:26.053040 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:26.053013    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" path="/var/lib/kubelet/pods/82254d34-de69-41c8-8678-2cdbc930eec1/volumes"
Jul 23 04:26:29.459980 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:29.459945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:30.813016 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:30.812984    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:35.905009 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:35.904961    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.004741 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.004709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.176018 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.175923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.403368 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.403333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.547617 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.547584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.648759 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.648729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:36.760562 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:36.760525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:38.397855 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:38.397798    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:42.702255 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:42.702176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:42.772251 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:42.772227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:42.851559 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:42.851528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:42.948545 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:42.948507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:43.075398 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:43.075362    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:48.963480 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:48.963446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:49.045295 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:49.045261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:55.793730 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:55.793679    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:56.103632 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:56.103554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:56.353274 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:56.353244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:58.614768 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.614724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:58.715417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.715383    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:58.814983 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.814920    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:26:58.815363 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.815343    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" containerName="maas-api"
Jul 23 04:26:58.815453 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.815365    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" containerName="maas-api"
Jul 23 04:26:58.815936 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.815905    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82254d34-de69-41c8-8678-2cdbc930eec1" containerName="maas-api"
Jul 23 04:26:58.821055 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.820422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.824051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.823511    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e0e3ac7b-serving-cert\""
Jul 23 04:26:58.839987 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.839961    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:26:58.873475 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.873399    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:58.878559 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.878535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.878659 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.878586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5m9g\" (UniqueName: \"kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.878659 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.878622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.972915 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.972862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:58.979208 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.979188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.979326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.979231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5m9g\" (UniqueName: \"kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.979326 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.979294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.979937 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.979914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.981842 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.981814    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:58.991150 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:58.991126    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5m9g\" (UniqueName: \"kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g\") pod \"maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:59.133843 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:59.133768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:26:59.274361 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:59.274335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:26:59.275967 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:26:59.275936    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea12ef9_760c_48b9_9386_fb5c84680aaa.slice/crio-b27bc932c607ce5119040ed08a4f8a322c9c06bc8fa18f1e41ed5631d63ba8fe WatchSource:0}: Error finding container b27bc932c607ce5119040ed08a4f8a322c9c06bc8fa18f1e41ed5631d63ba8fe: Status 404 returned error can't find the container with id b27bc932c607ce5119040ed08a4f8a322c9c06bc8fa18f1e41ed5631d63ba8fe
Jul 23 04:26:59.277248 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:59.277227    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:26:59.437230 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:59.437158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:26:59.534940 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:26:59.534907    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:27:00.121940 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:00.121907    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" event={"ID":"6ea12ef9-760c-48b9-9386-fb5c84680aaa","Type":"ContainerStarted","Data":"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"}
Jul 23 04:27:00.122333 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:00.121963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" event={"ID":"6ea12ef9-760c-48b9-9386-fb5c84680aaa","Type":"ContainerStarted","Data":"b27bc932c607ce5119040ed08a4f8a322c9c06bc8fa18f1e41ed5631d63ba8fe"}
Jul 23 04:27:00.122333 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:00.122048    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:27:00.143892 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:00.143837    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" podStartSLOduration=1.761687469 podStartE2EDuration="2.143795961s" totalImagesPullingTime="382.108492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:27:00.143047429 +0000 UTC m=+1902.604191103" watchObservedRunningTime="2026-07-23 04:27:00.143795961 +0000 UTC m=+1902.604939631"
Jul 23 04:27:03.107546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:03.107514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:27:06.132371 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:06.132341    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:27:10.036745 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:10.036682    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:27:51.992596 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:51.992564    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:27:51.993121 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:51.992782    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" podUID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" containerName="maas-api" containerID="cri-o://f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6" gracePeriod=30
Jul 23 04:27:52.031752 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.031723    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:27:52.247756 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.247695    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:27:52.321197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.321169    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca\") pod \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") "
Jul 23 04:27:52.321327 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.321258    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5m9g\" (UniqueName: \"kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g\") pod \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") "
Jul 23 04:27:52.321327 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.321302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls\") pod \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\" (UID: \"6ea12ef9-760c-48b9-9386-fb5c84680aaa\") "
Jul 23 04:27:52.321502 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.321481    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ea12ef9-760c-48b9-9386-fb5c84680aaa" (UID: "6ea12ef9-760c-48b9-9386-fb5c84680aaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:27:52.323188 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.323155    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g" (OuterVolumeSpecName: "kube-api-access-w5m9g") pod "6ea12ef9-760c-48b9-9386-fb5c84680aaa" (UID: "6ea12ef9-760c-48b9-9386-fb5c84680aaa"). InnerVolumeSpecName "kube-api-access-w5m9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:27:52.323323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.323304    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ea12ef9-760c-48b9-9386-fb5c84680aaa" (UID: "6ea12ef9-760c-48b9-9386-fb5c84680aaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:27:52.374616 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.374582    2582 generic.go:356] "Generic (PLEG): container finished" podID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" containerID="f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6" exitCode=0
Jul 23 04:27:52.374771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.374628    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"
Jul 23 04:27:52.374771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.374674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" event={"ID":"6ea12ef9-760c-48b9-9386-fb5c84680aaa","Type":"ContainerDied","Data":"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"}
Jul 23 04:27:52.374771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.374728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k" event={"ID":"6ea12ef9-760c-48b9-9386-fb5c84680aaa","Type":"ContainerDied","Data":"b27bc932c607ce5119040ed08a4f8a322c9c06bc8fa18f1e41ed5631d63ba8fe"}
Jul 23 04:27:52.374771 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.374752    2582 scope.go:117] "RemoveContainer" containerID="f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"
Jul 23 04:27:52.384622 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.384601    2582 scope.go:117] "RemoveContainer" containerID="f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"
Jul 23 04:27:52.384888 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:27:52.384871    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6\": container with ID starting with f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6 not found: ID does not exist" containerID="f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"
Jul 23 04:27:52.384945 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.384896    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6"} err="failed to get container status \"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6\": rpc error: code = NotFound desc = could not find container \"f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6\": container with ID starting with f125254c9048b94bd7c1ed1574af849880d893e4e3c279ed14c5395f9e0b91f6 not found: ID does not exist"
Jul 23 04:27:52.398055 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.398032    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:27:52.402968 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.402948    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e0e3ac7b-f768897-rrx7k"]
Jul 23 04:27:52.422359 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.422337    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea12ef9-760c-48b9-9386-fb5c84680aaa-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:27:52.422359 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.422357    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5m9g\" (UniqueName: \"kubernetes.io/projected/6ea12ef9-760c-48b9-9386-fb5c84680aaa-kube-api-access-w5m9g\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:27:52.422456 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:52.422367    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ea12ef9-760c-48b9-9386-fb5c84680aaa-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:27:54.052859 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:54.052828    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" path="/var/lib/kubelet/pods/6ea12ef9-760c-48b9-9386-fb5c84680aaa/volumes"
Jul 23 04:27:56.597034 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:56.596999    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:27:58.823874 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:27:58.823841    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:03.904269 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:03.904234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:12.220280 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:12.220199    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:15.832925 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:15.832895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:15.932976 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:15.932951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:39.949973 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:39.949938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:41.184539 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:41.184507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:46.886438 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:46.886408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:47.092901 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:47.092861    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:47.239823 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:47.239776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:47.446458 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:47.446426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:48.509577 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:48.509546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:48.610037 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:48.610004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:48.708201 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:48.708163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:49.453749 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:49.453713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:49.549749 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:49.549718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:49.649495 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:49.649465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:51.035421 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:51.035384    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:56.950510 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:56.950476    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:57.058695 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:57.058665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:28:57.647284 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:28:57.647249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:01.173524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:01.173487    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:01.278793 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:01.278764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:04.283616 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:04.283586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:04.388081 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:04.388048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:04.485734 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:04.485700    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:15.005649 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:15.005613    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:15.347264 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:15.347182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:36.312796 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:36.312758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:42.314049 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:42.313968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:42.401661 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:42.401626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:42.654937 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:42.654850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:42.985978 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:42.985951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:43.086110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:43.086079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:45.564106 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.564075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:45.653591 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.653550    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:29:45.654233 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.654210    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" containerName="maas-api"
Jul 23 04:29:45.654314 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.654237    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" containerName="maas-api"
Jul 23 04:29:45.654371 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.654346    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea12ef9-760c-48b9-9386-fb5c84680aaa" containerName="maas-api"
Jul 23 04:29:45.657751 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.657731    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.658300 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.658098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:45.660613 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.660435    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-30964d-serving-cert\""
Jul 23 04:29:45.668390 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.668341    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:29:45.730888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.730863    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:45.761223 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.761192    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.761355 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.761229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjpg9\" (UniqueName: \"kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.761355 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.761265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.831037 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.830932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:45.862369 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.862341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.862511 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.862378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjpg9\" (UniqueName: \"kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.862511 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.862417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.863072 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.863051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.864733 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.864715    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.871127 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.871104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjpg9\" (UniqueName: \"kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9\") pod \"maas-api-e2e-inf-a-30964d-86577dc65f-79frl\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:45.971579 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:45.971557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:46.251114 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.251082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:46.314502 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.314306    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:29:46.316212 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.316185    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:46.317315 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:29:46.317287    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ad9239_0313_44bc_ada7_0c8590525740.slice/crio-b2984be9f56af24acbb92145e3da97002ec7ad7fc781805aaf5369cc58b1c56e WatchSource:0}: Error finding container b2984be9f56af24acbb92145e3da97002ec7ad7fc781805aaf5369cc58b1c56e: Status 404 returned error can't find the container with id b2984be9f56af24acbb92145e3da97002ec7ad7fc781805aaf5369cc58b1c56e
Jul 23 04:29:46.933916 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.933882    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" event={"ID":"d7ad9239-0313-44bc-ada7-0c8590525740","Type":"ContainerStarted","Data":"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"}
Jul 23 04:29:46.934417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.933933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" event={"ID":"d7ad9239-0313-44bc-ada7-0c8590525740","Type":"ContainerStarted","Data":"b2984be9f56af24acbb92145e3da97002ec7ad7fc781805aaf5369cc58b1c56e"}
Jul 23 04:29:46.934417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.934034    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:46.952516 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:46.952465    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" podStartSLOduration=1.515681007 podStartE2EDuration="1.952451855s" totalImagesPullingTime="436.770848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:29:46.95105654 +0000 UTC m=+2069.412200231" watchObservedRunningTime="2026-07-23 04:29:46.952451855 +0000 UTC m=+2069.413595526"
Jul 23 04:29:49.988015 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:49.987985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:50.304794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.304702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:50.579681 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.579601    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:29:50.583311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.583282    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.585998 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.585977    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c7abdc-openshift-default-dockercfg-94jl9\""
Jul 23 04:29:50.594311 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.594288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:29:50.609184 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.609158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:50.706148 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706296    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llkgv\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706515 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706515 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706426    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.706515 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.706449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llkgv\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807697 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807697 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807697 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.807697 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.807658    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.808067 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.808042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.808209 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.808128    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.809703 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.809684    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.809790 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.809773    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.815561 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.815543    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.815713 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.815694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llkgv\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv\") pod \"e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") " pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:50.897351 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:50.897286    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:51.040769 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.040747    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:29:51.043105 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:29:51.043072    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f232b6_87c5_4d17_af95_013fd08b53d6.slice/crio-b48f6d863bf226c9426ec6e4ce5976eadbd8c622e712262de0ba49d863de956d WatchSource:0}: Error finding container b48f6d863bf226c9426ec6e4ce5976eadbd8c622e712262de0ba49d863de956d: Status 404 returned error can't find the container with id b48f6d863bf226c9426ec6e4ce5976eadbd8c622e712262de0ba49d863de956d
Jul 23 04:29:51.045387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.045356    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:29:51.045474 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.045422    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:29:51.045474 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.045454    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:29:51.126355 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.126320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:51.961415 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.961375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerStarted","Data":"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"}
Jul 23 04:29:51.961577 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.961433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerStarted","Data":"b48f6d863bf226c9426ec6e4ce5976eadbd8c622e712262de0ba49d863de956d"}
Jul 23 04:29:51.984337 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:51.984291    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podStartSLOduration=1.984275844 podStartE2EDuration="1.984275844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:29:51.982502467 +0000 UTC m=+2074.443646164" watchObservedRunningTime="2026-07-23 04:29:51.984275844 +0000 UTC m=+2074.445419516"
Jul 23 04:29:52.898166 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:52.898130    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:52.902934 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:52.902911    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:52.945283 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:52.945260    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:29:52.966633 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:52.966609    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:52.967327 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:52.967311    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:29:53.919075 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:53.919041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.014146 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.014101    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:29:54.017351 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.017319    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.021924 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.021861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.024773 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.024741    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c7abdc-serving-cert\""
Jul 23 04:29:54.039198 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.039168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:29:54.093550 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.093523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.137403 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.137377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.137541 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.137455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.137541 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.137491    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qkq9\" (UniqueName: \"kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.238060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.238035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.238197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.238070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkq9\" (UniqueName: \"kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.238197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.238101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.238946 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.238922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.240574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.240551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.246586 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.246561    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qkq9\" (UniqueName: \"kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9\") pod \"maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.269869 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.269844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.338564 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.338542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.487794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.487627    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:29:54.490251 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:29:54.490180    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19fbfc1d_e3b2_462b_815b_203643943dd8.slice/crio-bccc9b02a1dfad75340d03e80a24c28a6f6d70a4f6b47b950158ddb27f442726 WatchSource:0}: Error finding container bccc9b02a1dfad75340d03e80a24c28a6f6d70a4f6b47b950158ddb27f442726: Status 404 returned error can't find the container with id bccc9b02a1dfad75340d03e80a24c28a6f6d70a4f6b47b950158ddb27f442726
Jul 23 04:29:54.618257 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.618225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.719161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.719128    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:29:54.977568 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.977537    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" event={"ID":"19fbfc1d-e3b2-462b-815b-203643943dd8","Type":"ContainerStarted","Data":"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"}
Jul 23 04:29:54.978038 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.977596    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" event={"ID":"19fbfc1d-e3b2-462b-815b-203643943dd8","Type":"ContainerStarted","Data":"bccc9b02a1dfad75340d03e80a24c28a6f6d70a4f6b47b950158ddb27f442726"}
Jul 23 04:29:54.978038 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.977771    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:29:54.996440 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:54.996396    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" podStartSLOduration=1.657218238 podStartE2EDuration="1.996382055s" totalImagesPullingTime="339.163817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:29:54.994845172 +0000 UTC m=+2077.455988865" watchObservedRunningTime="2026-07-23 04:29:54.996382055 +0000 UTC m=+2077.457525726"
Jul 23 04:29:58.443488 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:29:58.443450    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:00.988697 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:00.988669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:30:03.189184 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:03.189155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:07.103167 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:07.103130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:07.203748 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:07.203717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:09.007984 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:09.007941    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 04:30:09.008471 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:09.008011    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 04:30:11.067988 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.067953    2582 generic.go:356] "Generic (PLEG): container finished" podID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerID="54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab" exitCode=0
Jul 23 04:30:11.068458 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.068008    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerDied","Data":"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"}
Jul 23 04:30:11.068458 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.068358    2582 scope.go:117] "RemoveContainer" containerID="54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"
Jul 23 04:30:11.069342 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.069312    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:30:11.069396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.069382    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:30:11.069438 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:11.069431    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:30:12.074823 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:12.074772    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerStarted","Data":"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"}
Jul 23 04:30:12.897876 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:12.897827    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:30:12.898121 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:12.898096    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 04:30:12.898184 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:12.898166    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 04:30:13.898149 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:13.898116    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 04:30:13.898606 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:13.898165    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 04:30:14.898018 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:14.897983    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 04:30:14.898183 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:14.898036    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 04:30:15.442005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:15.441953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:15.542307 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:15.542274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:30:15.901580 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:15.901541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:30:15.902020 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:15.901856    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:30:15.902419 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:15.902403    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:30:27.422269 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:30:27.422236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:13.145265 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:13.145222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:14.028275 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:14.028246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:14.733553 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:14.733519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:14.826646 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:14.826618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:24.616643 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.616609    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:31:24.617150 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.616861    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" podUID="d7ad9239-0313-44bc-ada7-0c8590525740" containerName="maas-api" containerID="cri-o://8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19" gracePeriod=30
Jul 23 04:31:24.644830 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.644777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:24.876123 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.876060    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:31:24.934738 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.934705    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjpg9\" (UniqueName: \"kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9\") pod \"d7ad9239-0313-44bc-ada7-0c8590525740\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") "
Jul 23 04:31:24.934738 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.934738    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls\") pod \"d7ad9239-0313-44bc-ada7-0c8590525740\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") "
Jul 23 04:31:24.934962 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.934776    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca\") pod \"d7ad9239-0313-44bc-ada7-0c8590525740\" (UID: \"d7ad9239-0313-44bc-ada7-0c8590525740\") "
Jul 23 04:31:24.935157 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.935127    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7ad9239-0313-44bc-ada7-0c8590525740" (UID: "d7ad9239-0313-44bc-ada7-0c8590525740"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:31:24.936741 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.936718    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9" (OuterVolumeSpecName: "kube-api-access-gjpg9") pod "d7ad9239-0313-44bc-ada7-0c8590525740" (UID: "d7ad9239-0313-44bc-ada7-0c8590525740"). InnerVolumeSpecName "kube-api-access-gjpg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:31:24.937253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:24.937226    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7ad9239-0313-44bc-ada7-0c8590525740" (UID: "d7ad9239-0313-44bc-ada7-0c8590525740"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:31:25.036437 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.036414    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjpg9\" (UniqueName: \"kubernetes.io/projected/d7ad9239-0313-44bc-ada7-0c8590525740-kube-api-access-gjpg9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:25.036527 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.036437    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ad9239-0313-44bc-ada7-0c8590525740-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:25.036527 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.036457    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ad9239-0313-44bc-ada7-0c8590525740-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:25.440484 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.440449    2582 generic.go:356] "Generic (PLEG): container finished" podID="d7ad9239-0313-44bc-ada7-0c8590525740" containerID="8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19" exitCode=0
Jul 23 04:31:25.440650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.440499    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"
Jul 23 04:31:25.440650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.440542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" event={"ID":"d7ad9239-0313-44bc-ada7-0c8590525740","Type":"ContainerDied","Data":"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"}
Jul 23 04:31:25.440650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.440595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl" event={"ID":"d7ad9239-0313-44bc-ada7-0c8590525740","Type":"ContainerDied","Data":"b2984be9f56af24acbb92145e3da97002ec7ad7fc781805aaf5369cc58b1c56e"}
Jul 23 04:31:25.440650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.440618    2582 scope.go:117] "RemoveContainer" containerID="8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"
Jul 23 04:31:25.449988 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.449972    2582 scope.go:117] "RemoveContainer" containerID="8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"
Jul 23 04:31:25.450208 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:31:25.450189    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19\": container with ID starting with 8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19 not found: ID does not exist" containerID="8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"
Jul 23 04:31:25.450274 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.450218    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19"} err="failed to get container status \"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19\": rpc error: code = NotFound desc = could not find container \"8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19\": container with ID starting with 8b311f4b9782e44b200527b4bcb1e18d7607970153148d57776743342844fa19 not found: ID does not exist"
Jul 23 04:31:25.461215 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.461192    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:31:25.464671 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:25.464651    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30964d-86577dc65f-79frl"]
Jul 23 04:31:26.053417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:26.053381    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ad9239-0313-44bc-ada7-0c8590525740" path="/var/lib/kubelet/pods/d7ad9239-0313-44bc-ada7-0c8590525740/volumes"
Jul 23 04:31:29.267032 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:29.266996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:29.673881 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:29.673766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:35.589069 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:35.589034    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:36.634423 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:36.634389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:36.739950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:36.739872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:36.840737 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:36.840703    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:37.829740 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:37.829706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:46.729125 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:46.729086    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:31:46.729608 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:46.729366    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" podUID="19fbfc1d-e3b2-462b-815b-203643943dd8" containerName="maas-api" containerID="cri-o://a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e" gracePeriod=30
Jul 23 04:31:46.759276 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:46.759244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:47.169781 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.169755    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:31:47.315595 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.315523    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls\") pod \"19fbfc1d-e3b2-462b-815b-203643943dd8\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") "
Jul 23 04:31:47.315595 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.315553    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qkq9\" (UniqueName: \"kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9\") pod \"19fbfc1d-e3b2-462b-815b-203643943dd8\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") "
Jul 23 04:31:47.315595 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.315571    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca\") pod \"19fbfc1d-e3b2-462b-815b-203643943dd8\" (UID: \"19fbfc1d-e3b2-462b-815b-203643943dd8\") "
Jul 23 04:31:47.316028 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.315999    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19fbfc1d-e3b2-462b-815b-203643943dd8" (UID: "19fbfc1d-e3b2-462b-815b-203643943dd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:31:47.317601 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.317568    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9" (OuterVolumeSpecName: "kube-api-access-2qkq9") pod "19fbfc1d-e3b2-462b-815b-203643943dd8" (UID: "19fbfc1d-e3b2-462b-815b-203643943dd8"). InnerVolumeSpecName "kube-api-access-2qkq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:31:47.317698 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.317655    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19fbfc1d-e3b2-462b-815b-203643943dd8" (UID: "19fbfc1d-e3b2-462b-815b-203643943dd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:31:47.416958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.416929    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19fbfc1d-e3b2-462b-815b-203643943dd8-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:47.416958 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.416957    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qkq9\" (UniqueName: \"kubernetes.io/projected/19fbfc1d-e3b2-462b-815b-203643943dd8-kube-api-access-2qkq9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:47.417118 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.416973    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fbfc1d-e3b2-462b-815b-203643943dd8-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:31:47.552611 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.552579    2582 generic.go:356] "Generic (PLEG): container finished" podID="19fbfc1d-e3b2-462b-815b-203643943dd8" containerID="a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e" exitCode=0
Jul 23 04:31:47.552754 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.552622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" event={"ID":"19fbfc1d-e3b2-462b-815b-203643943dd8","Type":"ContainerDied","Data":"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"}
Jul 23 04:31:47.552754 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.552638    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"
Jul 23 04:31:47.552754 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.552666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk" event={"ID":"19fbfc1d-e3b2-462b-815b-203643943dd8","Type":"ContainerDied","Data":"bccc9b02a1dfad75340d03e80a24c28a6f6d70a4f6b47b950158ddb27f442726"}
Jul 23 04:31:47.552754 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.552687    2582 scope.go:117] "RemoveContainer" containerID="a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"
Jul 23 04:31:47.562297 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.562279    2582 scope.go:117] "RemoveContainer" containerID="a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"
Jul 23 04:31:47.562533 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:31:47.562514    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e\": container with ID starting with a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e not found: ID does not exist" containerID="a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"
Jul 23 04:31:47.562583 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.562542    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e"} err="failed to get container status \"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e\": rpc error: code = NotFound desc = could not find container \"a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e\": container with ID starting with a5fcaae03bc31615ac28f13b48c5b7b0fa55446ccd3cdfcbd0c0a86bba83746e not found: ID does not exist"
Jul 23 04:31:47.576407 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.576352    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:31:47.584663 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:47.584641    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c7abdc-5944867cb4-gvcqk"]
Jul 23 04:31:48.052668 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:48.052640    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19fbfc1d-e3b2-462b-815b-203643943dd8" path="/var/lib/kubelet/pods/19fbfc1d-e3b2-462b-815b-203643943dd8/volumes"
Jul 23 04:31:51.295380 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:51.295346    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:51.709312 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:51.709228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:57.669657 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:57.669619    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:57.772407 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:57.772372    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:57.922436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:57.922357    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:31:57.922612 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:57.922584    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" containerID="cri-o://8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff" gracePeriod=30
Jul 23 04:31:57.941883 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:57.941851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:58.173325 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:58.173248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:58.509552 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:58.509520    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:58.911689 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:58.911611    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:31:59.013091 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:31:59.013057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:00.904097 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:00.904053    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 04:32:00.904452 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:00.904109    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 04:32:02.284389 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.284348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:02.385222 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.385182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:02.386436 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386405    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:02.386842 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386828    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ad9239-0313-44bc-ada7-0c8590525740" containerName="maas-api"
Jul 23 04:32:02.386891 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386844    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ad9239-0313-44bc-ada7-0c8590525740" containerName="maas-api"
Jul 23 04:32:02.386891 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386856    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19fbfc1d-e3b2-462b-815b-203643943dd8" containerName="maas-api"
Jul 23 04:32:02.386891 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386862    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="19fbfc1d-e3b2-462b-815b-203643943dd8" containerName="maas-api"
Jul 23 04:32:02.386991 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386940    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="19fbfc1d-e3b2-462b-815b-203643943dd8" containerName="maas-api"
Jul 23 04:32:02.386991 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.386951    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ad9239-0313-44bc-ada7-0c8590525740" containerName="maas-api"
Jul 23 04:32:02.390005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.389988    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.392593 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.392577    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b88d1b-serving-cert\""
Jul 23 04:32:02.401453 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.401428    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:02.460898 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.460871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:02.550211 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.550116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.550370 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.550237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.550370 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.550274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rsdv\" (UniqueName: \"kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.564468 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.564446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:02.651538 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.651507    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.651538 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.651540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rsdv\" (UniqueName: \"kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.651730 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.651598    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.652324 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.652296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.654161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.654137    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.659612 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.659587    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rsdv\" (UniqueName: \"kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv\") pod \"maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.701048 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.701020    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:02.841496 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.841342    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:02.845274 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:32:02.845245    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d58e58_9dc7_4096_977b_da5ee0704d43.slice/crio-ff4c8b2f1787e526908d028e27276ae34cbc7bc9d3170456c17d4be8abecb2f7 WatchSource:0}: Error finding container ff4c8b2f1787e526908d028e27276ae34cbc7bc9d3170456c17d4be8abecb2f7: Status 404 returned error can't find the container with id ff4c8b2f1787e526908d028e27276ae34cbc7bc9d3170456c17d4be8abecb2f7
Jul 23 04:32:02.847095 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.847075    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 04:32:02.954706 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:02.954680    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:03.054547 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.054516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:03.161851 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.161820    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:32:03.256080 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256052    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256080 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256085    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256133    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256181    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256205    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256316    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256345    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256386    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llkgv\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv\") pod \"f4f232b6-87c5-4d17-af95-013fd08b53d6\" (UID: \"f4f232b6-87c5-4d17-af95-013fd08b53d6\") "
Jul 23 04:32:03.256506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256409    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:32:03.256506 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:32:03.256776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256669    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:32:03.256776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256709    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f232b6-87c5-4d17-af95-013fd08b53d6-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.256776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256731    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.256965 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.256823    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:32:03.257216 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.257193    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:32:03.258385 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.258359    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 04:32:03.258483 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.258430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:32:03.258483 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.258449    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:32:03.258668 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.258647    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv" (OuterVolumeSpecName: "kube-api-access-llkgv") pod "f4f232b6-87c5-4d17-af95-013fd08b53d6" (UID: "f4f232b6-87c5-4d17-af95-013fd08b53d6"). InnerVolumeSpecName "kube-api-access-llkgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:32:03.358110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358078    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llkgv\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-kube-api-access-llkgv\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358104    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358110 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358114    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358123    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358131    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358139    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.358638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.358147    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f232b6-87c5-4d17-af95-013fd08b53d6-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:03.636290 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.636253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" event={"ID":"f3d58e58-9dc7-4096-977b-da5ee0704d43","Type":"ContainerStarted","Data":"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"}
Jul 23 04:32:03.636482 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.636411    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:03.636482 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.636443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" event={"ID":"f3d58e58-9dc7-4096-977b-da5ee0704d43","Type":"ContainerStarted","Data":"ff4c8b2f1787e526908d028e27276ae34cbc7bc9d3170456c17d4be8abecb2f7"}
Jul 23 04:32:03.638095 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.638070    2582 generic.go:356] "Generic (PLEG): container finished" podID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerID="8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff" exitCode=0
Jul 23 04:32:03.638205 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.638110    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"
Jul 23 04:32:03.638205 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.638154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerDied","Data":"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"}
Jul 23 04:32:03.638282 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.638207    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv" event={"ID":"f4f232b6-87c5-4d17-af95-013fd08b53d6","Type":"ContainerDied","Data":"b48f6d863bf226c9426ec6e4ce5976eadbd8c622e712262de0ba49d863de956d"}
Jul 23 04:32:03.638282 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.638235    2582 scope.go:117] "RemoveContainer" containerID="8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"
Jul 23 04:32:03.648013 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.647996    2582 scope.go:117] "RemoveContainer" containerID="54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"
Jul 23 04:32:03.657546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.657527    2582 scope.go:117] "RemoveContainer" containerID="8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"
Jul 23 04:32:03.657815 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:32:03.657785    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff\": container with ID starting with 8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff not found: ID does not exist" containerID="8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"
Jul 23 04:32:03.657889 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.657774    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" podStartSLOduration=1.125182754 podStartE2EDuration="1.657759374s" totalImagesPullingTime="532.57662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:32:03.656182987 +0000 UTC m=+2206.117326660" watchObservedRunningTime="2026-07-23 04:32:03.657759374 +0000 UTC m=+2206.118903040"
Jul 23 04:32:03.657889 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.657828    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff"} err="failed to get container status \"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff\": rpc error: code = NotFound desc = could not find container \"8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff\": container with ID starting with 8c0ffd9cd429fa79c6baab695b587c0818e5e081021a1f663b26211f8d0d98ff not found: ID does not exist"
Jul 23 04:32:03.657889 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.657861    2582 scope.go:117] "RemoveContainer" containerID="54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"
Jul 23 04:32:03.658104 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:32:03.658088    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab\": container with ID starting with 54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab not found: ID does not exist" containerID="54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"
Jul 23 04:32:03.658145 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.658108    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab"} err="failed to get container status \"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab\": rpc error: code = NotFound desc = could not find container \"54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab\": container with ID starting with 54ab27524498994b5ecec1ee8d6cca67ef337b0679e37974351fd6e86e174dab not found: ID does not exist"
Jul 23 04:32:03.670563 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.670539    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:32:03.676002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:03.675977    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c7abdc-openshift-default-7f6db68c95-r52rv"]
Jul 23 04:32:04.052574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:04.052541    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" path="/var/lib/kubelet/pods/f4f232b6-87c5-4d17-af95-013fd08b53d6/volumes"
Jul 23 04:32:06.774717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:06.774684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:07.256384 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.256350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:07.536168 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.536086    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:32:07.536695 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.536672    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.536882 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.536868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.537005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.536988    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.537005 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.537004    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.537197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.537109    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.537197 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.537126    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f232b6-87c5-4d17-af95-013fd08b53d6" containerName="istio-proxy"
Jul 23 04:32:07.542097 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.542078    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.545243 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.545226    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a4e7cf-openshift-default-dockercfg-4vp6s\""
Jul 23 04:32:07.554320 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.554296    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:32:07.581538 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.581513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:07.601860 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.601826    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.601942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.601886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.601942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.601907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.601942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.601927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.602078 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.601983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.602078 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.602063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.602174 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.602098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.602230 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.602174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.602282 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.602225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz8z9\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703131 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703098    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703161    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703302 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703304    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz8z9\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703509 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703702 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703702 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703660    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703785 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703770    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.703893 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.703875    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.704252 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.704231    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.705615 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.705591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.705877 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.705853    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.713344 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.713317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz8z9\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.714108 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.714085    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token\") pod \"e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") " pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:07.854169 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:07.854099    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:08.007374 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.007347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:32:08.010198 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:32:08.010168    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18596ef8_9b65_4244_8724_5ce2a80571e2.slice/crio-c6e3870041c5dfe635a9048769e5e7eef9b10f61a5aca8811aa99690af9f3264 WatchSource:0}: Error finding container c6e3870041c5dfe635a9048769e5e7eef9b10f61a5aca8811aa99690af9f3264: Status 404 returned error can't find the container with id c6e3870041c5dfe635a9048769e5e7eef9b10f61a5aca8811aa99690af9f3264
Jul 23 04:32:08.012496 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.012466    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:08.012556 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.012530    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:08.012639 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.012562    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:08.125554 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.125489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:08.665573 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.665536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerStarted","Data":"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"}
Jul 23 04:32:08.665761 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.665594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerStarted","Data":"c6e3870041c5dfe635a9048769e5e7eef9b10f61a5aca8811aa99690af9f3264"}
Jul 23 04:32:08.688308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.688259    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podStartSLOduration=1.688244968 podStartE2EDuration="1.688244968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:32:08.687238672 +0000 UTC m=+2211.148382364" watchObservedRunningTime="2026-07-23 04:32:08.688244968 +0000 UTC m=+2211.149388640"
Jul 23 04:32:08.855120 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:08.855080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:09.649420 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:09.649393    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:09.859424 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:09.859394    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:10.677472 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.677436    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:10.678427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.678407    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:10.715634 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.715600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:10.802417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.802386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:10.896692 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.896660    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:32:10.900234 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.900216    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:10.902607 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.902589    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a4e7cf-serving-cert\""
Jul 23 04:32:10.911219 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.911157    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:32:10.927938 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.927880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:10.927938 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.927923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:10.928078 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.927944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjfwd\" (UniqueName: \"kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:10.929322 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:10.929301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:11.029298 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.029264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.029473 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.029321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.029473 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.029357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjfwd\" (UniqueName: \"kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.029944 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.029923    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.031851 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.031827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.038262 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.038238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjfwd\" (UniqueName: \"kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd\") pod \"maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.128322 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.128297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:11.214454 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.214387    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:11.381231 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.380466    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:32:11.387448 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:32:11.387408    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb03e86_a2d5_45b2_9155_93a8cd11e60b.slice/crio-eb0847e038a8aa50c55267f9e13dad8a42d3cfec62a51e3b64bf75c3705fa326 WatchSource:0}: Error finding container eb0847e038a8aa50c55267f9e13dad8a42d3cfec62a51e3b64bf75c3705fa326: Status 404 returned error can't find the container with id eb0847e038a8aa50c55267f9e13dad8a42d3cfec62a51e3b64bf75c3705fa326
Jul 23 04:32:11.411323 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.411298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:11.512056 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.512032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:11.683316 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:11.683279    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" event={"ID":"afb03e86-a2d5-45b2-9155-93a8cd11e60b","Type":"ContainerStarted","Data":"eb0847e038a8aa50c55267f9e13dad8a42d3cfec62a51e3b64bf75c3705fa326"}
Jul 23 04:32:12.689883 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:12.689844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" event={"ID":"afb03e86-a2d5-45b2-9155-93a8cd11e60b","Type":"ContainerStarted","Data":"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"}
Jul 23 04:32:12.712158 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:12.712111    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" podStartSLOduration=2.115826534 podStartE2EDuration="2.712097975s" totalImagesPullingTime="596.271441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:32:12.709911041 +0000 UTC m=+2215.171054737" watchObservedRunningTime="2026-07-23 04:32:12.712097975 +0000 UTC m=+2215.173241667"
Jul 23 04:32:13.695351 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:13.695320    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:15.100686 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:15.100653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:19.705943 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:19.705910    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:32:23.643524 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:23.643489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:23.745417 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:23.745330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:26.718946 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:26.718903    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 04:32:26.719312 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:26.718988    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 04:32:27.773554 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.773521    2582 generic.go:356] "Generic (PLEG): container finished" podID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerID="f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432" exitCode=0
Jul 23 04:32:27.774002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.773594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerDied","Data":"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"}
Jul 23 04:32:27.774002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.773956    2582 scope.go:117] "RemoveContainer" containerID="f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"
Jul 23 04:32:27.774917 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.774890    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:27.774994 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.774957    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:27.774994 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:27.774988    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 23 04:32:28.780768 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:28.780730    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerStarted","Data":"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"}
Jul 23 04:32:28.854433 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:28.854383    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:28.854926 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:28.854879    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 23 04:32:28.855195 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:28.855144    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 23 04:32:29.854557 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:29.854522    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 23 04:32:29.855153 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:29.855119    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 23 04:32:30.854511 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:30.854477    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 23 04:32:30.854819 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:30.854763    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 23 04:32:31.854691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:31.854652    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 23 04:32:31.854921 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:31.854724    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 23 04:32:32.858338 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:32.858307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:32.858772 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:32.858637    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:32.859339 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:32.859317    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:32:33.236878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:33.236833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:33.337435 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:33.337401    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:38.545461 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.545431    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:38.546022 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.545650    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" podUID="f3d58e58-9dc7-4096-977b-da5ee0704d43" containerName="maas-api" containerID="cri-o://b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254" gracePeriod=30
Jul 23 04:32:38.562511 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.562479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:38.808597 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.808532    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:38.839259 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.839231    2582 generic.go:356] "Generic (PLEG): container finished" podID="f3d58e58-9dc7-4096-977b-da5ee0704d43" containerID="b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254" exitCode=0
Jul 23 04:32:38.839390 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.839295    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"
Jul 23 04:32:38.839390 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.839312    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" event={"ID":"f3d58e58-9dc7-4096-977b-da5ee0704d43","Type":"ContainerDied","Data":"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"}
Jul 23 04:32:38.839390 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.839372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf" event={"ID":"f3d58e58-9dc7-4096-977b-da5ee0704d43","Type":"ContainerDied","Data":"ff4c8b2f1787e526908d028e27276ae34cbc7bc9d3170456c17d4be8abecb2f7"}
Jul 23 04:32:38.839554 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.839398    2582 scope.go:117] "RemoveContainer" containerID="b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"
Jul 23 04:32:38.849588 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.849561    2582 scope.go:117] "RemoveContainer" containerID="b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"
Jul 23 04:32:38.849913 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:32:38.849887    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254\": container with ID starting with b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254 not found: ID does not exist" containerID="b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"
Jul 23 04:32:38.849998 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.849920    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254"} err="failed to get container status \"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254\": rpc error: code = NotFound desc = could not find container \"b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254\": container with ID starting with b20fefc75927b9d504e7fd86d0dbba27342dd48041787e3349e9c64d81b38254 not found: ID does not exist"
Jul 23 04:32:38.863341 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.863317    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls\") pod \"f3d58e58-9dc7-4096-977b-da5ee0704d43\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") "
Jul 23 04:32:38.863444 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.863392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rsdv\" (UniqueName: \"kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv\") pod \"f3d58e58-9dc7-4096-977b-da5ee0704d43\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") "
Jul 23 04:32:38.863444 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.863433    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca\") pod \"f3d58e58-9dc7-4096-977b-da5ee0704d43\" (UID: \"f3d58e58-9dc7-4096-977b-da5ee0704d43\") "
Jul 23 04:32:38.863794 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.863770    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3d58e58-9dc7-4096-977b-da5ee0704d43" (UID: "f3d58e58-9dc7-4096-977b-da5ee0704d43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:32:38.865260 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.865240    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3d58e58-9dc7-4096-977b-da5ee0704d43" (UID: "f3d58e58-9dc7-4096-977b-da5ee0704d43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:32:38.865499 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.865481    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv" (OuterVolumeSpecName: "kube-api-access-6rsdv") pod "f3d58e58-9dc7-4096-977b-da5ee0704d43" (UID: "f3d58e58-9dc7-4096-977b-da5ee0704d43"). InnerVolumeSpecName "kube-api-access-6rsdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:32:38.965094 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.965066    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rsdv\" (UniqueName: \"kubernetes.io/projected/f3d58e58-9dc7-4096-977b-da5ee0704d43-kube-api-access-6rsdv\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:38.965094 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.965091    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d58e58-9dc7-4096-977b-da5ee0704d43-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:38.965220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:38.965106    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d58e58-9dc7-4096-977b-da5ee0704d43-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:32:39.162254 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:39.162224    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:39.167017 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:39.166992    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b88d1b-69bfdd4dc5-dtxwf"]
Jul 23 04:32:40.052461 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:40.052426    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d58e58-9dc7-4096-977b-da5ee0704d43" path="/var/lib/kubelet/pods/f3d58e58-9dc7-4096-977b-da5ee0704d43/volumes"
Jul 23 04:32:43.223712 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:43.223675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:43.620265 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:43.620233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:50.059541 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:50.059508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:50.165449 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:50.165410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:50.376299 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:50.376202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:50.642591 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:50.642498    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:32:51.790672 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:32:51.790624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:06.246925 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.246893    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:33:06.247310 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.247096    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" podUID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" containerName="maas-api" containerID="cri-o://598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a" gracePeriod=30
Jul 23 04:33:06.270154 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.270130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:06.501319 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.501263    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:33:06.588332 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.588306    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjfwd\" (UniqueName: \"kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd\") pod \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") "
Jul 23 04:33:06.588453 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.588396    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca\") pod \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") "
Jul 23 04:33:06.588453 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.588412    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls\") pod \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\" (UID: \"afb03e86-a2d5-45b2-9155-93a8cd11e60b\") "
Jul 23 04:33:06.588706 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.588681    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "afb03e86-a2d5-45b2-9155-93a8cd11e60b" (UID: "afb03e86-a2d5-45b2-9155-93a8cd11e60b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:33:06.590374 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.590349    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd" (OuterVolumeSpecName: "kube-api-access-vjfwd") pod "afb03e86-a2d5-45b2-9155-93a8cd11e60b" (UID: "afb03e86-a2d5-45b2-9155-93a8cd11e60b"). InnerVolumeSpecName "kube-api-access-vjfwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:33:06.590608 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.590590    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "afb03e86-a2d5-45b2-9155-93a8cd11e60b" (UID: "afb03e86-a2d5-45b2-9155-93a8cd11e60b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:33:06.689336 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.689300    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjfwd\" (UniqueName: \"kubernetes.io/projected/afb03e86-a2d5-45b2-9155-93a8cd11e60b-kube-api-access-vjfwd\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:06.689336 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.689332    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb03e86-a2d5-45b2-9155-93a8cd11e60b-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:06.689336 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.689342    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb03e86-a2d5-45b2-9155-93a8cd11e60b-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:06.979073 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.979046    2582 generic.go:356] "Generic (PLEG): container finished" podID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" containerID="598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a" exitCode=0
Jul 23 04:33:06.979220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.979092    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"
Jul 23 04:33:06.979220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.979098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" event={"ID":"afb03e86-a2d5-45b2-9155-93a8cd11e60b","Type":"ContainerDied","Data":"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"}
Jul 23 04:33:06.979220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.979142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4" event={"ID":"afb03e86-a2d5-45b2-9155-93a8cd11e60b","Type":"ContainerDied","Data":"eb0847e038a8aa50c55267f9e13dad8a42d3cfec62a51e3b64bf75c3705fa326"}
Jul 23 04:33:06.979220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.979164    2582 scope.go:117] "RemoveContainer" containerID="598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"
Jul 23 04:33:06.988717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.988698    2582 scope.go:117] "RemoveContainer" containerID="598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"
Jul 23 04:33:06.989086 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:33:06.989064    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a\": container with ID starting with 598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a not found: ID does not exist" containerID="598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"
Jul 23 04:33:06.989160 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:06.989098    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a"} err="failed to get container status \"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a\": rpc error: code = NotFound desc = could not find container \"598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a\": container with ID starting with 598b0cab3b1945a6992b3618eebc14e05be6b91d01f8ac0dc2c125419938ce4a not found: ID does not exist"
Jul 23 04:33:07.004056 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:07.004035    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:33:07.008622 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:07.008602    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4e7cf-74f7cfbdfc-tgtv4"]
Jul 23 04:33:08.053935 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:08.053882    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" path="/var/lib/kubelet/pods/afb03e86-a2d5-45b2-9155-93a8cd11e60b/volumes"
Jul 23 04:33:10.876888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:10.876858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:11.216044 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:11.215964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:17.040856 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.040818    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:17.142064 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.142028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:17.348554 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.348466    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:33:17.348740 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.348695    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" containerID="cri-o://4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea" gracePeriod=30
Jul 23 04:33:17.357776 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.357751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:17.655516 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.655433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:17.861064 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.861028    2582 patch_prober.go:28] interesting pod/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 04:33:17.861223 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.861094    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 04:33:17.948857 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:17.948767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:18.273971 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:18.273934    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:18.432439 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:18.432405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:21.760243 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.760205    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:21.871288 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.871256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:21.871642 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.871621    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:33:21.872070 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872053    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" containerName="maas-api"
Jul 23 04:33:21.872162 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872072    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" containerName="maas-api"
Jul 23 04:33:21.872162 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872096    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d58e58-9dc7-4096-977b-da5ee0704d43" containerName="maas-api"
Jul 23 04:33:21.872162 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872105    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d58e58-9dc7-4096-977b-da5ee0704d43" containerName="maas-api"
Jul 23 04:33:21.872316 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872201    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb03e86-a2d5-45b2-9155-93a8cd11e60b" containerName="maas-api"
Jul 23 04:33:21.872316 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.872220    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d58e58-9dc7-4096-977b-da5ee0704d43" containerName="maas-api"
Jul 23 04:33:21.876773 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.876753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:21.880114 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.880098    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5fa859-serving-cert\""
Jul 23 04:33:21.887377 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.887355    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:33:21.962995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:21.962962    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:22.019392 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.019307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.019546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.019392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.019546 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.019441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm7mq\" (UniqueName: \"kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.120037 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.119999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.120209 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.120098    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.120209 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.120147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm7mq\" (UniqueName: \"kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.120753 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.120725    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.122775 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.122745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.139441 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.139416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm7mq\" (UniqueName: \"kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq\") pod \"maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.181850 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.181792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:22.187888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.187862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:22.330452 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.330429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:33:22.333064 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:33:22.333024    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55622d50_5eec_4029_925d_feb37e65bad4.slice/crio-407bd079f4e3e1c9260c1ef9f2db46c9e77c850bc4733b426607716d91d4f213 WatchSource:0}: Error finding container 407bd079f4e3e1c9260c1ef9f2db46c9e77c850bc4733b426607716d91d4f213: Status 404 returned error can't find the container with id 407bd079f4e3e1c9260c1ef9f2db46c9e77c850bc4733b426607716d91d4f213
Jul 23 04:33:22.490990 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.490960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:22.596187 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.596165    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:33:22.597673 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.597652    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:22.726016 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.725988    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726016 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726145    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726178    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726200    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz8z9\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726237 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726227    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726550 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726272    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket\") pod \"18596ef8-9b65-4244-8724-5ce2a80571e2\" (UID: \"18596ef8-9b65-4244-8724-5ce2a80571e2\") "
Jul 23 04:33:22.726609 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726577    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:33:22.726676 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726655    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:33:22.726749 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726726    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:33:22.726964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.726927    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:33:22.727153 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.727135    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:33:22.728421 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.728390    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 04:33:22.728669 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.728642    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:33:22.728778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.728679    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9" (OuterVolumeSpecName: "kube-api-access-pz8z9") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "kube-api-access-pz8z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:33:22.728778 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.728738    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "18596ef8-9b65-4244-8724-5ce2a80571e2" (UID: "18596ef8-9b65-4244-8724-5ce2a80571e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:33:22.827829 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827789    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.827829 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827831    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827841    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827849    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18596ef8-9b65-4244-8724-5ce2a80571e2-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827858    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827867    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827880    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz8z9\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-kube-api-access-pz8z9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827889    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18596ef8-9b65-4244-8724-5ce2a80571e2-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:22.828253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:22.827897    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18596ef8-9b65-4244-8724-5ce2a80571e2-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:33:23.060227 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.060198    2582 generic.go:356] "Generic (PLEG): container finished" podID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerID="4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea" exitCode=0
Jul 23 04:33:23.060406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.060237    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"
Jul 23 04:33:23.060406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.060283    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerDied","Data":"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"}
Jul 23 04:33:23.060406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.060335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl" event={"ID":"18596ef8-9b65-4244-8724-5ce2a80571e2","Type":"ContainerDied","Data":"c6e3870041c5dfe635a9048769e5e7eef9b10f61a5aca8811aa99690af9f3264"}
Jul 23 04:33:23.060406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.060357    2582 scope.go:117] "RemoveContainer" containerID="4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"
Jul 23 04:33:23.062175 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.062147    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" event={"ID":"55622d50-5eec-4029-925d-feb37e65bad4","Type":"ContainerStarted","Data":"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"}
Jul 23 04:33:23.062308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.062200    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" event={"ID":"55622d50-5eec-4029-925d-feb37e65bad4","Type":"ContainerStarted","Data":"407bd079f4e3e1c9260c1ef9f2db46c9e77c850bc4733b426607716d91d4f213"}
Jul 23 04:33:23.062308 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.062233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:23.072378 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.072359    2582 scope.go:117] "RemoveContainer" containerID="f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"
Jul 23 04:33:23.081127 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.081111    2582 scope.go:117] "RemoveContainer" containerID="4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"
Jul 23 04:33:23.081358 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:33:23.081340    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea\": container with ID starting with 4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea not found: ID does not exist" containerID="4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"
Jul 23 04:33:23.081402 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.081367    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea"} err="failed to get container status \"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea\": rpc error: code = NotFound desc = could not find container \"4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea\": container with ID starting with 4f9e7bff136dad4f4b04cdc18d09543b7ee9c9af0e73c29862ade5e12b8db7ea not found: ID does not exist"
Jul 23 04:33:23.081402 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.081384    2582 scope.go:117] "RemoveContainer" containerID="f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"
Jul 23 04:33:23.081606 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:33:23.081585    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432\": container with ID starting with f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432 not found: ID does not exist" containerID="f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"
Jul 23 04:33:23.081693 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.081610    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432"} err="failed to get container status \"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432\": rpc error: code = NotFound desc = could not find container \"f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432\": container with ID starting with f36bc511d5783b3b885a854c970196dd278d33be7fa031f65457b5b330f1d432 not found: ID does not exist"
Jul 23 04:33:23.096410 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.096342    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" podStartSLOduration=1.6344580579999999 podStartE2EDuration="2.096329636s" totalImagesPullingTime="461.871578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:33:23.09413432 +0000 UTC m=+2285.555277993" watchObservedRunningTime="2026-07-23 04:33:23.096329636 +0000 UTC m=+2285.557473307"
Jul 23 04:33:23.127370 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.127346    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:33:23.138233 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:23.138211    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a4e7cf-openshift-default-5cc4d989cc-zt4dl"]
Jul 23 04:33:24.054167 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:24.054137    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" path="/var/lib/kubelet/pods/18596ef8-9b65-4244-8724-5ce2a80571e2/volumes"
Jul 23 04:33:26.236431 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:26.236402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:26.589236 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:26.589156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:26.946515 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:26.946438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:27.118294 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:27.118264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:29.073010 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.072980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:33:29.936254 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.936220    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:29.985215 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985179    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:33:29.985623 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985605    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.985709 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985625    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.985709 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985642    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.985709 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985652    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.985896 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985776    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.985896 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.985789    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18596ef8-9b65-4244-8724-5ce2a80571e2" containerName="istio-proxy"
Jul 23 04:33:29.989304 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.989279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:29.993484 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:29.993459    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ab39e0-serving-cert\""
Jul 23 04:33:30.010046 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.010020    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:33:30.082433 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.082404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:30.087845 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.087796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxlt\" (UniqueName: \"kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.087950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.087860    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.087950 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.087908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.188427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.188345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:30.188427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.188365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxlt\" (UniqueName: \"kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.188427 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.188408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.188657 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.188456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.189083 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.189066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.190734 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.190717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.204772 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.204752    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxlt\" (UniqueName: \"kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt\") pod \"maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.300881 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.300851    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:30.437690 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.437659    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:33:30.438955 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:33:30.438881    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b809fa4_dadc_4255_98df_3331d1943e5f.slice/crio-06a387ac54a736373873c5381bf8d1edcdc46efb790fe435a5fd8de7674b48f5 WatchSource:0}: Error finding container 06a387ac54a736373873c5381bf8d1edcdc46efb790fe435a5fd8de7674b48f5: Status 404 returned error can't find the container with id 06a387ac54a736373873c5381bf8d1edcdc46efb790fe435a5fd8de7674b48f5
Jul 23 04:33:30.692346 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.692251    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:30.806888 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:30.806843    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:31.104352 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:31.104319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" event={"ID":"3b809fa4-dadc-4255-98df-3331d1943e5f","Type":"ContainerStarted","Data":"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"}
Jul 23 04:33:31.104764 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:31.104374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" event={"ID":"3b809fa4-dadc-4255-98df-3331d1943e5f","Type":"ContainerStarted","Data":"06a387ac54a736373873c5381bf8d1edcdc46efb790fe435a5fd8de7674b48f5"}
Jul 23 04:33:31.104764 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:31.104422    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:31.130147 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:31.130106    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" podStartSLOduration=1.787220813 podStartE2EDuration="2.130091752s" totalImagesPullingTime="342.870939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:33:31.130012874 +0000 UTC m=+2293.591156571" watchObservedRunningTime="2026-07-23 04:33:31.130091752 +0000 UTC m=+2293.591235498"
Jul 23 04:33:34.334949 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:34.334868    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:37.114670 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:37.114643    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:33:44.673639 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:44.673606    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:44.773924 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:44.773886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:52.900364 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:52.900314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:52.998396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:52.998363    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:33:53.406854 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:33:53.406797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:34:01.999714 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:34:01.999672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:34:02.097824 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:34:02.097782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:34:05.197309 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:34:05.197274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:34:35.563448 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:34:35.563412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:35:44.897947 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:35:44.897857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:35:51.486942 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:35:51.486912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:36:08.280866 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:36:08.280829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:36:08.379929 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:36:08.379898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:36:47.550455 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:36:47.550413    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:36:49.032346 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:36:49.032309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:37:00.226263 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:37:00.226231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:37:32.008693 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:37:32.008662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:37:40.837259 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:37:40.837222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:23.592139 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:23.592097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:24.570777 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:24.570748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:33.401951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.401919    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:38:33.405430 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.405414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.408077 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.408058    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"openshift-service-ca.crt\""
Jul 23 04:38:33.408269 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.408255    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"kube-root-ca.crt\""
Jul 23 04:38:33.409341 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.409322    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"default-dockercfg-fgrvl\""
Jul 23 04:38:33.409433 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.409388    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5fa859\"/\"rate-test-model-5fa859-kserve-self-signed-certs\""
Jul 23 04:38:33.416031 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.416009    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:38:33.427254 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.427223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:33.533147 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.533324 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fnjq\" (UniqueName: \"kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.533324 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.533324 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533297    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.533494 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533346    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.533494 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.533382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634525 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634496    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634650 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634737 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fnjq\" (UniqueName: \"kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634737 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.634951 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.634737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.635070 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.635042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.635070 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.635067    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.635255 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.635110    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.636734 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.636715    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.637050 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.637033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.643945 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.643921    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fnjq\" (UniqueName: \"kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq\") pod \"rate-test-model-5fa859-kserve-55c85cfbb7-97r8w\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") " pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.715570 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.715494    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:33.852669 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:33.852640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:38:33.854672 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:38:33.854640    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a6d1287_2525_4f82_9aef_c250876d2fd8.slice/crio-bef03a64ac29adb4eaa07df03ce463d8a9285f6a3c1f57849437a0c0752acf71 WatchSource:0}: Error finding container bef03a64ac29adb4eaa07df03ce463d8a9285f6a3c1f57849437a0c0752acf71: Status 404 returned error can't find the container with id bef03a64ac29adb4eaa07df03ce463d8a9285f6a3c1f57849437a0c0752acf71
Jul 23 04:38:34.589820 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:34.589765    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" event={"ID":"3a6d1287-2525-4f82-9aef-c250876d2fd8","Type":"ContainerStarted","Data":"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"}
Jul 23 04:38:34.590230 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:34.589841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" event={"ID":"3a6d1287-2525-4f82-9aef-c250876d2fd8","Type":"ContainerStarted","Data":"bef03a64ac29adb4eaa07df03ce463d8a9285f6a3c1f57849437a0c0752acf71"}
Jul 23 04:38:34.619179 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:34.619134    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" podStartSLOduration=1.6191217839999998 podStartE2EDuration="1.619121784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:38:34.618966993 +0000 UTC m=+2597.080110664" watchObservedRunningTime="2026-07-23 04:38:34.619121784 +0000 UTC m=+2597.080265455"
Jul 23 04:38:43.715924 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:43.715844    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:43.715924 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:43.715889    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:43.728429 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:43.728403    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:44.654745 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:44.654719    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:38:45.663150 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:45.663118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:45.767682 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:45.767655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:53.439768 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:53.439730    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:53.726766 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:53.726673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:38:53.829146 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:38:53.829109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:02.787330 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:02.787296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:02.888879 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:02.888842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:33.533638 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:33.533604    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:39:33.546060 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:33.546029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:34.332014 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:34.331978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:48.553863 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.553789    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" podUID="3a6d1287-2525-4f82-9aef-c250876d2fd8" containerName="main" containerID="cri-o://3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32" gracePeriod=45
Jul 23 04:39:48.802827 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.802788    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:39:48.966757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.966672    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a6d1287-2525-4f82-9aef-c250876d2fd8" containerID="3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32" exitCode=0
Jul 23 04:39:48.966757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.966715    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"
Jul 23 04:39:48.966757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.966748    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" event={"ID":"3a6d1287-2525-4f82-9aef-c250876d2fd8","Type":"ContainerDied","Data":"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"}
Jul 23 04:39:48.967002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.966790    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w" event={"ID":"3a6d1287-2525-4f82-9aef-c250876d2fd8","Type":"ContainerDied","Data":"bef03a64ac29adb4eaa07df03ce463d8a9285f6a3c1f57849437a0c0752acf71"}
Jul 23 04:39:48.967002 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.966830    2582 scope.go:117] "RemoveContainer" containerID="3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"
Jul 23 04:39:48.975526 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.975511    2582 scope.go:117] "RemoveContainer" containerID="3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"
Jul 23 04:39:48.975779 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:39:48.975763    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32\": container with ID starting with 3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32 not found: ID does not exist" containerID="3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"
Jul 23 04:39:48.975848 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.975788    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32"} err="failed to get container status \"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32\": rpc error: code = NotFound desc = could not find container \"3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32\": container with ID starting with 3d1ea214d4804df24a5e54dceb7ceac9ab552da4cdb0cb84363caea6172a6f32 not found: ID does not exist"
Jul 23 04:39:48.989103 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989085    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fnjq\" (UniqueName: \"kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989181 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989111    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989181 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989139    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989294 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989353 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989296    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989364    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:48.989406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989376    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm\") pod \"3a6d1287-2525-4f82-9aef-c250876d2fd8\" (UID: \"3a6d1287-2525-4f82-9aef-c250876d2fd8\") "
Jul 23 04:39:48.989406 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989382    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:48.989606 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989582    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home" (OuterVolumeSpecName: "home") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:48.989738 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989724    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-home\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:48.989827 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989740    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-model-cache\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:48.989827 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.989751    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-tmp-dir\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:48.991318 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.991299    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm" (OuterVolumeSpecName: "dshm") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 04:39:48.991399 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.991317    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq" (OuterVolumeSpecName: "kube-api-access-7fnjq") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "kube-api-access-7fnjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:39:48.991399 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:48.991354    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a6d1287-2525-4f82-9aef-c250876d2fd8" (UID: "3a6d1287-2525-4f82-9aef-c250876d2fd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:39:49.090396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:49.090370    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a6d1287-2525-4f82-9aef-c250876d2fd8-tls-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.090396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:49.090392    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a6d1287-2525-4f82-9aef-c250876d2fd8-dshm\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.090396 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:49.090401    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fnjq\" (UniqueName: \"kubernetes.io/projected/3a6d1287-2525-4f82-9aef-c250876d2fd8-kube-api-access-7fnjq\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:39:49.288502 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:49.288473    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:39:49.292496 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:49.292474    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5fa859/rate-test-model-5fa859-kserve-55c85cfbb7-97r8w"]
Jul 23 04:39:50.053003 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:50.052974    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a6d1287-2525-4f82-9aef-c250876d2fd8" path="/var/lib/kubelet/pods/3a6d1287-2525-4f82-9aef-c250876d2fd8/volumes"
Jul 23 04:39:53.794357 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:53.794321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:39:53.898484 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:39:53.898456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:03.705529 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:03.705495    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:40:03.705982 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:03.705718    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" podUID="55622d50-5eec-4029-925d-feb37e65bad4" containerName="maas-api" containerID="cri-o://5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f" gracePeriod=30
Jul 23 04:40:03.730988 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:03.730944    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:03.966846 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:03.966748    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:40:04.006717 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.006690    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca\") pod \"55622d50-5eec-4029-925d-feb37e65bad4\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") "
Jul 23 04:40:04.006905 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.006744    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm7mq\" (UniqueName: \"kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq\") pod \"55622d50-5eec-4029-925d-feb37e65bad4\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") "
Jul 23 04:40:04.006905 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.006779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls\") pod \"55622d50-5eec-4029-925d-feb37e65bad4\" (UID: \"55622d50-5eec-4029-925d-feb37e65bad4\") "
Jul 23 04:40:04.007199 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.007162    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55622d50-5eec-4029-925d-feb37e65bad4" (UID: "55622d50-5eec-4029-925d-feb37e65bad4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:40:04.008964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.008939    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55622d50-5eec-4029-925d-feb37e65bad4" (UID: "55622d50-5eec-4029-925d-feb37e65bad4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:40:04.009054 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.008976    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq" (OuterVolumeSpecName: "kube-api-access-mm7mq") pod "55622d50-5eec-4029-925d-feb37e65bad4" (UID: "55622d50-5eec-4029-925d-feb37e65bad4"). InnerVolumeSpecName "kube-api-access-mm7mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:40:04.045703 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.045678    2582 generic.go:356] "Generic (PLEG): container finished" podID="55622d50-5eec-4029-925d-feb37e65bad4" containerID="5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f" exitCode=0
Jul 23 04:40:04.045830 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.045723    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"
Jul 23 04:40:04.045830 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.045753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" event={"ID":"55622d50-5eec-4029-925d-feb37e65bad4","Type":"ContainerDied","Data":"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"}
Jul 23 04:40:04.045830 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.045793    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d" event={"ID":"55622d50-5eec-4029-925d-feb37e65bad4","Type":"ContainerDied","Data":"407bd079f4e3e1c9260c1ef9f2db46c9e77c850bc4733b426607716d91d4f213"}
Jul 23 04:40:04.045948 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.045845    2582 scope.go:117] "RemoveContainer" containerID="5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"
Jul 23 04:40:04.055769 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.055750    2582 scope.go:117] "RemoveContainer" containerID="5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"
Jul 23 04:40:04.056085 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:40:04.056060    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f\": container with ID starting with 5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f not found: ID does not exist" containerID="5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"
Jul 23 04:40:04.056192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.056094    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f"} err="failed to get container status \"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f\": rpc error: code = NotFound desc = could not find container \"5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f\": container with ID starting with 5e63ed17f031cdbf2ebbb90f634d6cde3dcd147b1d82cc80d20831441b846c5f not found: ID does not exist"
Jul 23 04:40:04.070662 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.070634    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:40:04.074253 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.074234    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5fa859-556df8b47-g6c4d"]
Jul 23 04:40:04.108057 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.108037    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55622d50-5eec-4029-925d-feb37e65bad4-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:04.108057 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.108058    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm7mq\" (UniqueName: \"kubernetes.io/projected/55622d50-5eec-4029-925d-feb37e65bad4-kube-api-access-mm7mq\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:04.108171 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:04.108069    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55622d50-5eec-4029-925d-feb37e65bad4-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:06.052544 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:06.052512    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55622d50-5eec-4029-925d-feb37e65bad4" path="/var/lib/kubelet/pods/55622d50-5eec-4029-925d-feb37e65bad4/volumes"
Jul 23 04:40:08.339072 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:08.339036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:08.820207 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:08.820176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:15.268969 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:15.268934    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:15.372668 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:15.372638    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:15.636577 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:15.636495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:15.815626 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:15.815596    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:16.711933 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:16.711900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:16.808366 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:16.808332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:26.361440 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.361406    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:40:26.361938 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.361689    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" podUID="3b809fa4-dadc-4255-98df-3331d1943e5f" containerName="maas-api" containerID="cri-o://eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960" gracePeriod=30
Jul 23 04:40:26.386745 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.386712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:26.620518 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.620457    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:40:26.697055 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.697032    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cxlt\" (UniqueName: \"kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt\") pod \"3b809fa4-dadc-4255-98df-3331d1943e5f\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") "
Jul 23 04:40:26.697186 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.697075    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca\") pod \"3b809fa4-dadc-4255-98df-3331d1943e5f\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") "
Jul 23 04:40:26.697186 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.697137    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls\") pod \"3b809fa4-dadc-4255-98df-3331d1943e5f\" (UID: \"3b809fa4-dadc-4255-98df-3331d1943e5f\") "
Jul 23 04:40:26.697501 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.697474    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b809fa4-dadc-4255-98df-3331d1943e5f" (UID: "3b809fa4-dadc-4255-98df-3331d1943e5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 04:40:26.699058 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.699028    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b809fa4-dadc-4255-98df-3331d1943e5f" (UID: "3b809fa4-dadc-4255-98df-3331d1943e5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 04:40:26.699159 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.699083    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt" (OuterVolumeSpecName: "kube-api-access-5cxlt") pod "3b809fa4-dadc-4255-98df-3331d1943e5f" (UID: "3b809fa4-dadc-4255-98df-3331d1943e5f"). InnerVolumeSpecName "kube-api-access-5cxlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 04:40:26.798192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.798162    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cxlt\" (UniqueName: \"kubernetes.io/projected/3b809fa4-dadc-4255-98df-3331d1943e5f-kube-api-access-5cxlt\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:26.798192 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.798190    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b809fa4-dadc-4255-98df-3331d1943e5f-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:26.798337 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:26.798200    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b809fa4-dadc-4255-98df-3331d1943e5f-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Jul 23 04:40:27.162694 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.162662    2582 generic.go:356] "Generic (PLEG): container finished" podID="3b809fa4-dadc-4255-98df-3331d1943e5f" containerID="eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960" exitCode=0
Jul 23 04:40:27.162908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.162714    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"
Jul 23 04:40:27.162908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.162734    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" event={"ID":"3b809fa4-dadc-4255-98df-3331d1943e5f","Type":"ContainerDied","Data":"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"}
Jul 23 04:40:27.162908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.162787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl" event={"ID":"3b809fa4-dadc-4255-98df-3331d1943e5f","Type":"ContainerDied","Data":"06a387ac54a736373873c5381bf8d1edcdc46efb790fe435a5fd8de7674b48f5"}
Jul 23 04:40:27.162908 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.162834    2582 scope.go:117] "RemoveContainer" containerID="eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"
Jul 23 04:40:27.172220 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.172205    2582 scope.go:117] "RemoveContainer" containerID="eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"
Jul 23 04:40:27.172491 ip-10-0-138-244 kubenswrapper[2582]: E0723 04:40:27.172464    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960\": container with ID starting with eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960 not found: ID does not exist" containerID="eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"
Jul 23 04:40:27.172556 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.172500    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960"} err="failed to get container status \"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960\": rpc error: code = NotFound desc = could not find container \"eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960\": container with ID starting with eb5af90036f43e6f432777e960b564ea97e792e5c5f236fb87894de3e9011960 not found: ID does not exist"
Jul 23 04:40:27.183574 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.183551    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:40:27.187293 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:27.187274    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ab39e0-77544ddc6b-n4bpl"]
Jul 23 04:40:28.054031 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:28.054003    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b809fa4-dadc-4255-98df-3331d1943e5f" path="/var/lib/kubelet/pods/3b809fa4-dadc-4255-98df-3331d1943e5f/volumes"
Jul 23 04:40:31.000477 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:31.000439    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:31.404639 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:31.404558    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:37.903519 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:37.903485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:38.003853 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:38.003824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:38.195535 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:38.195456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:38.421758 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:38.421721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:44.453365 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:44.453330    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b76d45f8-lz4rv_b40e973b-7433-421f-90a4-c9ef2d8253c4/manager/0.log"
Jul 23 04:40:44.568526 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:44.568500    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cm5xv_b2b76864-a24b-4d0c-90e3-6a55d353b877/server/0.log"
Jul 23 04:40:44.691051 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:44.691021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q7clh_281d5502-941f-4894-a8c9-139c52c30775/manager/0.log"
Jul 23 04:40:44.964095 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:44.964067    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-d2wsp_64196160-52a7-4314-80d1-bcd328e53c90/copy-manifests/0.log"
Jul 23 04:40:44.972363 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:44.972339    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-d2wsp_64196160-52a7-4314-80d1-bcd328e53c90/manager/0.log"
Jul 23 04:40:45.457414 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:45.457378    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cd75f5b7c-khrhj_34edca31-e28c-49a9-b1a2-ae1bd5933ddb/maas-api/0.log"
Jul 23 04:40:45.820556 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:45.820533    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jhql7_af113fbf-f451-42a0-ab57-ea807e0895b9/postgres/0.log"
Jul 23 04:40:47.407692 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:47.407668    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z6q5q_02110674-69f4-475e-afa6-323a61f4d622/manager/0.log"
Jul 23 04:40:47.637995 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:47.637971    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c8lkx_b3e66f3c-1a25-4d86-bdf8-5452063d92b5/registry-server/0.log"
Jul 23 04:40:47.769966 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:47.769938    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lcgsk_103d5bc4-827e-43ed-be5c-22d1d4764ee3/manager/0.log"
Jul 23 04:40:48.516911 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:48.516883    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-469qn_444e5f9c-fb5f-4d56-b778-fcaa96ff2b27/discovery/0.log"
Jul 23 04:40:48.755444 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:48.755412    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-qjb69_4f06d7ac-1278-477c-888e-4242ef8756f2/kube-auth-proxy/0.log"
Jul 23 04:40:48.873735 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:48.873662    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/1.log"
Jul 23 04:40:48.994383 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:48.994351    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-kpqs9_d512474f-d91b-4cc8-85ac-394d6580a0ef/payload-pre-processing/0.log"
Jul 23 04:40:49.115166 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:49.115138    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-srghr_96cdee91-9254-4db2-a07d-d4eef07df505/payload-processing/0.log"
Jul 23 04:40:49.610294 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:49.610266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6cp6f_06027ed6-cd34-48cf-abd8-85822b1c564c/main/0.log"
Jul 23 04:40:49.961293 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:49.961215    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f58mtt_1390e177-42ed-40f0-ba86-fd3bbb84df0f/main/0.log"
Jul 23 04:40:50.188188 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:50.188162    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-dfpnd_f6254b88-a0e7-4550-846d-c29e1b4e5842/main/0.log"
Jul 23 04:40:55.464720 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:55.464662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:55.557964 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:55.557937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log" limit=1024
Jul 23 04:40:58.390763 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:58.390728    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tzgz6_58af239e-1d0b-472a-8dc7-ddc26061a37a/global-pull-secret-syncer/0.log"
Jul 23 04:40:58.493539 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:58.493514    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8g4mn_dfaa5c1e-ed91-43d8-92c3-ceef77b14ec6/konnectivity-agent/0.log"
Jul 23 04:40:58.620405 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:40:58.620376    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-244.ec2.internal_11d946897901f8dd71884d5e0994bb9a/haproxy/0.log"
Jul 23 04:41:03.793757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:03.793723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z6q5q_02110674-69f4-475e-afa6-323a61f4d622/manager/0.log"
Jul 23 04:41:03.872691 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:03.872657    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c8lkx_b3e66f3c-1a25-4d86-bdf8-5452063d92b5/registry-server/0.log"
Jul 23 04:41:04.038207 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:04.038173    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lcgsk_103d5bc4-827e-43ed-be5c-22d1d4764ee3/manager/0.log"
Jul 23 04:41:06.128681 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:06.128652    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-69wkz_561f167a-69fe-4e68-9c6c-638b470a31a4/cluster-monitoring-operator/0.log"
Jul 23 04:41:06.498742 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:06.498719    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r59j8_3a25a26f-41c2-4540-9dac-5e6d1a32b1ec/node-exporter/0.log"
Jul 23 04:41:06.539379 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:06.539357    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r59j8_3a25a26f-41c2-4540-9dac-5e6d1a32b1ec/kube-rbac-proxy/0.log"
Jul 23 04:41:06.571926 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:06.571908    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r59j8_3a25a26f-41c2-4540-9dac-5e6d1a32b1ec/init-textfile/0.log"
Jul 23 04:41:08.481739 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.481708    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"]
Jul 23 04:41:08.482168 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482126    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6d1287-2525-4f82-9aef-c250876d2fd8" containerName="main"
Jul 23 04:41:08.482168 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482139    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6d1287-2525-4f82-9aef-c250876d2fd8" containerName="main"
Jul 23 04:41:08.482168 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482152    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b809fa4-dadc-4255-98df-3331d1943e5f" containerName="maas-api"
Jul 23 04:41:08.482168 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482158    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b809fa4-dadc-4255-98df-3331d1943e5f" containerName="maas-api"
Jul 23 04:41:08.482306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482174    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55622d50-5eec-4029-925d-feb37e65bad4" containerName="maas-api"
Jul 23 04:41:08.482306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482179    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="55622d50-5eec-4029-925d-feb37e65bad4" containerName="maas-api"
Jul 23 04:41:08.482306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482240    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b809fa4-dadc-4255-98df-3331d1943e5f" containerName="maas-api"
Jul 23 04:41:08.482306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482248    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6d1287-2525-4f82-9aef-c250876d2fd8" containerName="main"
Jul 23 04:41:08.482306 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.482257    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="55622d50-5eec-4029-925d-feb37e65bad4" containerName="maas-api"
Jul 23 04:41:08.485387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.485370    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.487946 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.487924    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czc5s\"/\"openshift-service-ca.crt\""
Jul 23 04:41:08.488063 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.487922    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czc5s\"/\"kube-root-ca.crt\""
Jul 23 04:41:08.488876 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.488860    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-czc5s\"/\"default-dockercfg-qqcf2\""
Jul 23 04:41:08.500832 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.496823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"]
Jul 23 04:41:08.525641 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.525614    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-sys\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.525757 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.525669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-proc\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.525878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.525853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-lib-modules\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.525987 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.525964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-podres\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.526044 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.526032    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b25tj\" (UniqueName: \"kubernetes.io/projected/90fcfe2a-e692-4504-b074-08ef5c28620c-kube-api-access-b25tj\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627387 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627364    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b25tj\" (UniqueName: \"kubernetes.io/projected/90fcfe2a-e692-4504-b074-08ef5c28620c-kube-api-access-b25tj\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-sys\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-proc\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-lib-modules\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-podres\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627523 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-sys\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627766 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-proc\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627766 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627626    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-podres\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.627766 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.627659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90fcfe2a-e692-4504-b074-08ef5c28620c-lib-modules\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.636238 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.636215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b25tj\" (UniqueName: \"kubernetes.io/projected/90fcfe2a-e692-4504-b074-08ef5c28620c-kube-api-access-b25tj\") pod \"perf-node-gather-daemonset-7k2xt\" (UID: \"90fcfe2a-e692-4504-b074-08ef5c28620c\") " pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.796165 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.796109    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:08.927009 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:08.926981    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"]
Jul 23 04:41:08.928986 ip-10-0-138-244 kubenswrapper[2582]: W0723 04:41:08.928955    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod90fcfe2a_e692_4504_b074_08ef5c28620c.slice/crio-a8e4f4266c9726f1c6fc7aa3ded635594843568ff2fc99d19817d70b84aa438c WatchSource:0}: Error finding container a8e4f4266c9726f1c6fc7aa3ded635594843568ff2fc99d19817d70b84aa438c: Status 404 returned error can't find the container with id a8e4f4266c9726f1c6fc7aa3ded635594843568ff2fc99d19817d70b84aa438c
Jul 23 04:41:09.366483 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.366445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt" event={"ID":"90fcfe2a-e692-4504-b074-08ef5c28620c","Type":"ContainerStarted","Data":"9925c766024e36529b117b83ddd4922c194e364a4e7525172212dbc4b036c2df"}
Jul 23 04:41:09.366657 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.366506    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt" event={"ID":"90fcfe2a-e692-4504-b074-08ef5c28620c","Type":"ContainerStarted","Data":"a8e4f4266c9726f1c6fc7aa3ded635594843568ff2fc99d19817d70b84aa438c"}
Jul 23 04:41:09.366657 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.366535    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:09.384457 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.384410    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt" podStartSLOduration=1.38439633 podStartE2EDuration="1.38439633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:09.382602315 +0000 UTC m=+2751.843745988" watchObservedRunningTime="2026-07-23 04:41:09.38439633 +0000 UTC m=+2751.845540002"
Jul 23 04:41:09.628172 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.628094    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f455bbfc8-9kwh7_ba118e9b-ff2c-4513-aeb3-4ea987feb0d1/console/0.log"
Jul 23 04:41:09.672252 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:09.672221    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-pl647_4d6a1218-fdb1-479a-b701-dfa362d94abc/download-server/0.log"
Jul 23 04:41:11.320878 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:11.320846    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9kq_d1646b1d-eac6-4790-a075-e03e632f44ed/dns/0.log"
Jul 23 04:41:11.349698 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:11.349673    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9kq_d1646b1d-eac6-4790-a075-e03e632f44ed/kube-rbac-proxy/0.log"
Jul 23 04:41:11.502242 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:11.502220    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dd5td_71ede43b-4d5c-4ee5-87c0-384f67a7509d/dns-node-resolver/0.log"
Jul 23 04:41:12.140875 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:12.140847    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nbptw_09f1bdd1-e10c-4253-8f7c-1606d3e0b58a/node-ca/0.log"
Jul 23 04:41:13.198949 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.198922    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-469qn_444e5f9c-fb5f-4d56-b778-fcaa96ff2b27/discovery/0.log"
Jul 23 04:41:13.251083 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.251053    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-qjb69_4f06d7ac-1278-477c-888e-4242ef8756f2/kube-auth-proxy/0.log"
Jul 23 04:41:13.279392 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.279365    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/0.log"
Jul 23 04:41:13.305161 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.305141    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/1.log"
Jul 23 04:41:13.335351 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.335331    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-kpqs9_d512474f-d91b-4cc8-85ac-394d6580a0ef/payload-pre-processing/0.log"
Jul 23 04:41:13.362572 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:13.362550    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-srghr_96cdee91-9254-4db2-a07d-d4eef07df505/payload-processing/0.log"
Jul 23 04:41:14.138244 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.138215    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xcmgc_f4f65d93-ddb5-4809-a5fc-cc584a12a728/serve-healthcheck-canary/0.log"
Jul 23 04:41:14.609050 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.609014    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-qmmfl_6b396972-7717-4089-83b5-9f856fa2cbe9/insights-operator/0.log"
Jul 23 04:41:14.609568 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.609548    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-qmmfl_6b396972-7717-4089-83b5-9f856fa2cbe9/insights-operator/1.log"
Jul 23 04:41:14.701625 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.701600    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-np2cq_a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b/kube-rbac-proxy/0.log"
Jul 23 04:41:14.726846 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.726825    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-np2cq_a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b/exporter/0.log"
Jul 23 04:41:14.748759 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:14.748740    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-np2cq_a17fc82a-a365-4a22-8b9a-fcbc45a3fa9b/extractor/0.log"
Jul 23 04:41:15.381011 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:15.380987    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-czc5s/perf-node-gather-daemonset-7k2xt"
Jul 23 04:41:17.584856 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:17.584822    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b76d45f8-lz4rv_b40e973b-7433-421f-90a4-c9ef2d8253c4/manager/0.log"
Jul 23 04:41:17.609185 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:17.609158    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cm5xv_b2b76864-a24b-4d0c-90e3-6a55d353b877/server/0.log"
Jul 23 04:41:17.677675 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:17.677644    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q7clh_281d5502-941f-4894-a8c9-139c52c30775/manager/0.log"
Jul 23 04:41:17.813028 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:17.813007    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-d2wsp_64196160-52a7-4314-80d1-bcd328e53c90/manager/0.log"
Jul 23 04:41:17.854067 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:17.854043    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5775946f86-d2wsp_64196160-52a7-4314-80d1-bcd328e53c90/copy-manifests/0.log"
Jul 23 04:41:19.392596 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.392565    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-469qn_444e5f9c-fb5f-4d56-b778-fcaa96ff2b27/discovery/0.log"
Jul 23 04:41:19.425093 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.425068    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-qjb69_4f06d7ac-1278-477c-888e-4242ef8756f2/kube-auth-proxy/0.log"
Jul 23 04:41:19.447084 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.447064    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/0.log"
Jul 23 04:41:19.471857 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.471798    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/1.log"
Jul 23 04:41:19.491520 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.491495    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-kpqs9_d512474f-d91b-4cc8-85ac-394d6580a0ef/payload-pre-processing/0.log"
Jul 23 04:41:19.509682 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:19.509658    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-srghr_96cdee91-9254-4db2-a07d-d4eef07df505/payload-processing/0.log"
Jul 23 04:41:20.854877 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.854850    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-469qn_444e5f9c-fb5f-4d56-b778-fcaa96ff2b27/discovery/0.log"
Jul 23 04:41:20.881590 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.881571    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c76cc57cd-qjb69_4f06d7ac-1278-477c-888e-4242ef8756f2/kube-auth-proxy/0.log"
Jul 23 04:41:20.904430 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.904404    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/0.log"
Jul 23 04:41:20.929851 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.929829    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lhgxx_198d3833-107a-4c9f-9dc4-10df2acee80d/istio-proxy/1.log"
Jul 23 04:41:20.953885 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.953863    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-kpqs9_d512474f-d91b-4cc8-85ac-394d6580a0ef/payload-pre-processing/0.log"
Jul 23 04:41:20.974222 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:20.974200    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-srghr_96cdee91-9254-4db2-a07d-d4eef07df505/payload-processing/0.log"
Jul 23 04:41:24.194689 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:24.194663    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-tzf8g_ec00ba70-a009-432c-8e9c-861caf24c734/migrator/0.log"
Jul 23 04:41:24.215108 ip-10-0-138-244 kubenswrapper[2582]: I0723 04:41:24.215081    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-tzf8g_ec00ba70-a009-432c-8e9c-861caf24c734/graceful-termination/0.log"