Jul 16 21:50:16.591522 ip-10-0-138-41 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.928637    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933396    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933407    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933410    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933413    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933416    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933419    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933421    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.029964 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933424    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933428    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933431    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933434    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933436    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933439    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933442    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933445    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933447    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933450    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933452    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933455    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933458    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933461    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933470    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933474    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933478    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933481    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933483    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.030817 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933486    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933488    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933491    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933494    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933496    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933499    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933501    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933504    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933506    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933509    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933512    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933514    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933517    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933519    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933522    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933524    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933527    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933530    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933533    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933535    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933538    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.032231 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933540    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933543    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933545    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933549    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933553    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933555    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933558    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933564    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933567    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933571    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933574    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933578    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933582    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933584    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933587    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933590    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933592    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933595    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933598    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.033072 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933600    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933603    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933605    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933608    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933610    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933613    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933616    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933618    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933621    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933624    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933626    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933629    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933632    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933634    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933637    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933639    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933642    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933645    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933647    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933649    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.033698 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933652    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933655    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933658    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933660    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933664    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933666    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933669    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933671    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933674    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933676    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933679    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933682    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.933684    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936494    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936505    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936509    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936512    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936515    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936518    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936521    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.035678 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936524    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936527    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936530    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936533    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936536    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936538    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936541    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936544    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936547    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936550    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936553    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936556    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936558    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936561    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936564    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936567    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936570    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936573    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936576    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.037224 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936579    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936582    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936584    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936588    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936591    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936594    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936597    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936599    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936602    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936605    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936608    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936610    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936613    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936615    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936618    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936622    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936626    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936629    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936632    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.039661 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936634    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936637    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936640    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936643    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936645    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936650    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936654    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936657    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936659    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936662    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936665    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936668    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936670    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936673    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936676    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936678    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936681    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936684    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936687    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.041868 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936689    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936692    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936694    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936697    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936700    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936702    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936705    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936708    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936711    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936713    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936716    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936719    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936721    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936724    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936727    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936729    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936732    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936737    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936740    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936743    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.046400 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936745    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936748    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936751    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936753    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936756    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936758    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936763    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936767    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936770    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936772    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936775    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936777    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936780    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936783    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.936786    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937155    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937160    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937163    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937165    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.047437 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937168    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937170    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937173    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937176    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937178    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937181    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937184    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937187    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937190    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937194    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937199    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937202    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937206    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937208    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937211    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937214    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937218    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937221    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937223    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.049617 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937226    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937229    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937231    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937234    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937236    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937239    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937242    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937244    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937247    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937268    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937271    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937274    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937277    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937279    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937282    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937284    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937287    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937289    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937292    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937295    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.051029 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937297    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937300    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937303    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937307    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937310    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937312    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937316    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937319    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937322    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937324    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937327    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937329    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937332    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937334    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937337    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937339    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937342    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937345    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937348    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937350    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.051907 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937353    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937357    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937360    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937364    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937367    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937370    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937372    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937375    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937377    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937380    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937382    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937385    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937388    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937391    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937394    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937397    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937400    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937402    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937405    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.052952 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937409    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937411    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937414    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937416    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937419    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937422    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937424    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937427    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937429    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937432    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937434    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937437    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937439    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937442    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937445    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937448    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.937450    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937522    2551 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937529    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937534    2551 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937540    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 21:50:17.053571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937546    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937550    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937557    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937562    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937565    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937568    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937571    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937576    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937579    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937582    2551 flags.go:64] FLAG: --cgroup-root=""
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937585    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937588    2551 flags.go:64] FLAG: --client-ca-file=""
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937597    2551 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937600    2551 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937605    2551 flags.go:64] FLAG: --cluster-domain=""
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937608    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937611    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937614    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937617    2551 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937625    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937628    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937631    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937634    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937637    2551 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 21:50:17.054594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937640    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937643    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937646    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937648    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937652    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937655    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937658    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937662    2551 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937665    2551 flags.go:64] FLAG: --enable-server="true"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937668    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937672    2551 flags.go:64] FLAG: --event-burst="100"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937675    2551 flags.go:64] FLAG: --event-qps="50"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937678    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937681    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937684    2551 flags.go:64] FLAG: --eviction-hard=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937689    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937692    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937696    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937699    2551 flags.go:64] FLAG: --eviction-soft=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937701    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937704    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937708    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937711    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937713    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937716    2551 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 21:50:17.055420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937719    2551 flags.go:64] FLAG: --feature-gates=""
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937723    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937726    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937730    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937733    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937736    2551 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937739    2551 flags.go:64] FLAG: --help="false"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937741    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937744    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937747    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937750    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937754    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937757    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937767    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937771    2551 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937775    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937778    2551 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937781    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937784    2551 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937787    2551 flags.go:64] FLAG: --kube-reserved=""
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937790    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937792    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937797    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937799    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 21:50:17.056234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937802    2551 flags.go:64] FLAG: --lock-file=""
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937805    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937808    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937811    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937816    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937820    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937823    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937826    2551 flags.go:64] FLAG: --logging-format="text"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937828    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937832    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937834    2551 flags.go:64] FLAG: --manifest-url=""
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937837    2551 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937841    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937844    2551 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937848    2551 flags.go:64] FLAG: --max-pods="110"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937851    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937854    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937856    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937859    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937862    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937865    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937868    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937875    2551 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937879    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937882    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 21:50:17.057060 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937885    2551 flags.go:64] FLAG: --pod-cidr=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937888    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937893    2551 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937896    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937899    2551 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937902    2551 flags.go:64] FLAG: --port="10250"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937907    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937910    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a0b4897b520e461d"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937913    2551 flags.go:64] FLAG: --qos-reserved=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937916    2551 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937919    2551 flags.go:64] FLAG: --register-node="true"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937922    2551 flags.go:64] FLAG: --register-with-taints=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937925    2551 flags.go:64] FLAG: --registry-burst="10"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937930    2551 flags.go:64] FLAG: --registry-qps="5"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937933    2551 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937935    2551 flags.go:64] FLAG: --reserved-memory=""
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937939    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937942    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937945    2551 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937948    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937951    2551 flags.go:64] FLAG: --runonce="false"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937954    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937957    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937959    2551 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937962    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 21:50:17.057899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937965    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937968    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937971    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937974    2551 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937976    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937981    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937983    2551 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937986    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937989    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937992    2551 flags.go:64] FLAG: --system-cgroups=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.937995    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938000    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938003    2551 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938007    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938012    2551 flags.go:64] FLAG: --tls-min-version=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938014    2551 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938017    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938020    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938023    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938026    2551 flags.go:64] FLAG: --v="2"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938030    2551 flags.go:64] FLAG: --version="false"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938034    2551 flags.go:64] FLAG: --vmodule=""
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938038    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.938041    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938159    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.058700 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938163    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938166    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938169    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938172    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938175    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938177    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938180    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938182    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938185    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938188    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938191    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938193    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938197    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938200    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938202    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938205    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938208    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938210    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938213    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938215    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.059424 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938219    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938221    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938224    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938226    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938229    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938232    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938235    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938237    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938240    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938243    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938246    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938267    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938270    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938273    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938277    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938281    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938284    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938287    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938290    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.060082 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938292    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938295    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938298    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938301    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938304    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938309    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938312    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938315    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938318    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938320    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938323    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938326    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938328    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938333    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938335    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938338    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938340    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938343    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938347    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938349    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.060627 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938353    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938357    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938360    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938363    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938365    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938368    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938371    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938373    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938376    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938378    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938381    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938384    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938386    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938389    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938391    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938393    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938396    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938401    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938404    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938406    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.061379 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938409    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938411    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938414    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938416    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938419    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938423    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938425    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938428    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938431    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938433    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938436    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938438    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938441    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938443    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938446    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938449    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938452    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938454    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.061956 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.938457    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.939103    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.944906    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.944921    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.944939    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.944945    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.944998    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945004    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945007    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945011    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945013    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945016    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945019    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945022    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945024    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.062488 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945027    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945030    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945033    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945035    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945038    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945041    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945043    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945046    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945049    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945052    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945055    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945058    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945060    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945063    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945065    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945068    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945071    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945073    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945076    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.062893 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945078    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945082    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945085    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945088    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945090    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945093    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945096    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945099    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945102    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945104    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945107    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945109    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945112    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945115    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945117    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945120    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945123    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945125    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945128    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945130    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.063579 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945133    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945135    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945138    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945141    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945144    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945146    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945149    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945151    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945154    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945156    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945159    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945162    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945164    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945167    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945171    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945174    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945178    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945182    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945185    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945188    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.125970 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945191    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.072768 ip-10-0-138-41 systemd[1]: Started Kubernetes Kubelet.
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945194    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945196    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945200    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945202    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945205    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945208    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945210    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945213    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945216    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945218    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945221    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945223    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945226    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945228    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945231    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945234    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945236    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945238    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.126938 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945242    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945246    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945263    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945267    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945270    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945272    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945277    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945280    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945284    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945287    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945290    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945292    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.945297    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945408    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945412    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:50:17.127967 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945415    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945417    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945423    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945426    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945428    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945431    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945433    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945436    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945438    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945441    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945444    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945447    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945449    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945452    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945455    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945459    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945463    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945466    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945469    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:50:17.128780 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945472    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945475    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945477    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945479    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945482    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945485    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945489    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945493    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945496    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945498    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945501    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945504    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945507    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945509    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945512    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945516    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945518    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945521    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945523    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945526    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:50:17.129883 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945529    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945531    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945534    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945537    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945541    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945544    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945546    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945549    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945552    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945554    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945557    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945559    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945562    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945564    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945567    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945569    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945573    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945576    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945578    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:50:17.130916 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945581    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945584    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945587    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945589    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945592    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945594    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945597    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945599    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945603    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945606    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945608    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945611    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945613    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945616    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945619    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945621    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945624    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945627    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945629    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:50:17.132184 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945632    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945634    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945637    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945639    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945642    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945644    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945647    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945649    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945651    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945654    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945658    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945660    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945663    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945665    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945668    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945671    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945673    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945676    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945679    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:50:17.133017 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:16.945681    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.945686    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.945795    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.947801    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.949038    2551 server.go:1019] "Starting client certificate rotation"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.949132    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.949163    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.951186    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.952802    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.962045    2551 log.go:25] "Validated CRI v1 runtime API"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.965152    2551 log.go:25] "Validated CRI v1 image API"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.966013    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 21:50:17.134043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.969805    2551 fs.go:135] Filesystem UUIDs: map[38d2f66f-5364-4205-a4cd-d13108d9823c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f16f8026-db32-42cf-9601-3005f86317ed:/dev/nvme0n1p3]
Jul 16 21:50:17.134659 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.969820    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.974434    2551 manager.go:216] Machine: {Timestamp:2026-07-16 21:50:16.973285512 +0000 UTC m=+0.275458198 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3124212 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c9442779e1818707ae4b0e79f4aad SystemUUID:ec2c9442-779e-1818-707a-e4b0e79f4aad BootID:b6af4ef8-3c42-4cb0-ab85-fe184dd7ae42 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:e9:aa:ff:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:e9:aa:ff:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:58:89:92:60:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.974529    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.974590    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.977357    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.977386    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-41.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.978073    2551 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.978085    2551 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 21:50:17.135280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.978095    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.978111    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:16.980444    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.007352    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qp4h5"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.016152    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qp4h5"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.023745    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060780    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060922    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060949    2551 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060958    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060971    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060981    2551 kubelet.go:398] "Adding apiserver pod source"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.060988    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.061325    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.061348    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.064643    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.066573    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.066617    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068146    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068161    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068167    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068173    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068184    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068193    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068203    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068208    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068214    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068219    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068227    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.068232    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.069595    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.069602    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.072314    2551 server.go:1268] "Started kubelet"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.073580    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.073780    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.074548    2551 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.075014    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.075047    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.076569    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.087981    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.091772    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.091947    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.082596    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-41.ec2.internal" not found
Jul 16 21:50:17.135633 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.092023    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-41.ec2.internal\" not found"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.092892    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.092934    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.092952    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.093113    2551 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.093121    2551 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.093462    2551 factory.go:55] Registering systemd factory
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.093477    2551 factory.go:223] Registration of the systemd container factory successfully
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.093794    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094042    2551 factory.go:153] Registering CRI-O factory
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094057    2551 factory.go:223] Registration of the crio container factory successfully
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094112    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094135    2551 factory.go:103] Registering Raw factory
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094150    2551 manager.go:1196] Started watching for new ooms in manager
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094596    2551 manager.go:349] Starting recovery of all containers
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.094691    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.100751    2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-41.ec2.internal\" not found" node="ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.103392    2551 manager.go:354] Recovery completed
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.105055    2551 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.109588    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.110912    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-41.ec2.internal" not found
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.111516    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.111532    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.111548    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112080    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112090    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112110    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112093    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112862    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112887    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.112899    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.114290    2551 policy_none.go:49] "None policy: Start"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.114303    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.114312    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 21:50:17.137306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.116272    2551 policy_none.go:47] "Start"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.164492    2551 manager.go:337] "Starting Device Plugin manager"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.164518    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.164527    2551 server.go:87] "Starting device plugin registration server"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.164685    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.164693    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.178683    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.178854    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.178863    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.179147    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.179180    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-41.ec2.internal\" not found"
Jul 16 21:50:17.252532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.181850    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-41.ec2.internal" not found
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.265609    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.266293    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.266309    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.266320    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.266345    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.274973    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.274992    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-41.ec2.internal\": node \"ip-10-0-138-41.ec2.internal\" not found"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.288142    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.289267    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.289308    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.289333    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:17.289368    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 21:50:17.351448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.291718    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:17.425196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.397779    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"]
Jul 16 21:50:17.425196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.400136    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.425196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.402197    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.448759    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.453519    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.461216    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.462047    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.498362    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74669e993430a0a3567c9a05c1f57949-config\") pod \"kube-apiserver-proxy-ip-10-0-138-41.ec2.internal\" (UID: \"74669e993430a0a3567c9a05c1f57949\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.498385    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.536882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.498403    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598845    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74669e993430a0a3567c9a05c1f57949-config\") pod \"kube-apiserver-proxy-ip-10-0-138-41.ec2.internal\" (UID: \"74669e993430a0a3567c9a05c1f57949\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598870    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598887    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598927    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598939    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74669e993430a0a3567c9a05c1f57949-config\") pod \"kube-apiserver-proxy-ip-10-0-138-41.ec2.internal\" (UID: \"74669e993430a0a3567c9a05c1f57949\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.658196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.598962    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fb1e5172fa4dc5ea0b1b27edef40bb69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal\" (UID: \"fb1e5172fa4dc5ea0b1b27edef40bb69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.838520 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.764188    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.838520 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.765355    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal"
Jul 16 21:50:17.950408 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.950367    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 21:50:17.950484 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.950469    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:50:17.950534 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.950506    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:50:17.950534 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:17.950510    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:50:18.031783 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.031739    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:45:17 +0000 UTC" deadline="2028-04-24 11:53:33.473262046 +0000 UTC"
Jul 16 21:50:18.031783 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.031778    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15542h3m15.441486609s"
Jul 16 21:50:18.063291 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.063270    2551 apiserver.go:52] "Watching apiserver"
Jul 16 21:50:18.071581 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.071562    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 21:50:18.071943 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.071920    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-nx9cb","openshift-multus/multus-additional-cni-plugins-dr7m8","openshift-multus/network-metrics-daemon-2m5v5","openshift-network-diagnostics/network-check-target-shstt","openshift-ovn-kubernetes/ovnkube-node-t77lx","kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw","openshift-cluster-node-tuning-operator/tuned-6zfwt","openshift-dns/node-resolver-n9tkf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal","openshift-multus/multus-6ppht","openshift-network-operator/iptables-alerter-487vh","kube-system/global-pull-secret-syncer-l9vjn","kube-system/konnectivity-agent-kvhc6"]
Jul 16 21:50:18.074595 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.074575    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.076937 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.076916    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.077359 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.077331    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 21:50:18.077448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.077400    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x5rkl\""
Jul 16 21:50:18.077448 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.077442    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.077741 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.077727    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.079318 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.079301    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.079389 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.079358    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 21:50:18.079451 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.079425    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:18.079506 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.079468    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 21:50:18.079700 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.079682    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.079763 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.079749    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 21:50:18.080016 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.080002    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c57pg\""
Jul 16 21:50:18.080157 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.080142    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.081646 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.081628    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:18.081709 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.081694    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:18.084145 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.084129    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.087031 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.087011    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.087132 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.087114    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.089541 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.089520    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 21:50:18.090064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.090048    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.090599 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.090578    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 21:50:18.090691 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.090633    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 21:50:18.090691 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.090665    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.092147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092076    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 21:50:18.092147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092097    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 21:50:18.092147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092115    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 21:50:18.092865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092115    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96xbq\""
Jul 16 21:50:18.092865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092440    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.092865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092497    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.092865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092557    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.093071 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.092977    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4bxd2\""
Jul 16 21:50:18.093586 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.093566    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-86dm6\""
Jul 16 21:50:18.093673 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.093599    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.093673 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.093608    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.095339 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.095309    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.096007 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.095990    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.096078 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.095994    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z9bbv\""
Jul 16 21:50:18.097687 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.097672    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 21:50:18.097687 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.097682    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.098035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.098021    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vjkpk\""
Jul 16 21:50:18.098916 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.098900    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.100192 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.100178    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.100253 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.100225    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:18.101231 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101218    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 21:50:18.101436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101418    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-log-socket\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.101497 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101450    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-serviceca\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.101497 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101480    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.101601 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101496    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.101601 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101525    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.101601 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101536    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cfdnc\""
Jul 16 21:50:18.101601 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101566    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhlzv\" (UniqueName: \"kubernetes.io/projected/e027d183-8aec-4901-b982-4b2c88a803a8-kube-api-access-jhlzv\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101616    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101665    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-sys\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101695    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-host\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101720    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-cni-binary-copy\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101753    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-etc-kubernetes\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.101798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-system-cni-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101811    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-env-overrides\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101836    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-device-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101861    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj59b\" (UniqueName: \"kubernetes.io/projected/5351fd34-a13a-4dc7-84a1-f68100b8c43c-kube-api-access-kj59b\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101886    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-config\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101910    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovn-node-metrics-cert\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101936    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-os-release\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101979    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.101997    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102015    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-system-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102019    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102058    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-multus\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.102083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102088    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-bin\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102106    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-conf\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102122    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102167    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysconfig\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102212    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-cnibin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102240    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-conf-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102306    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-netd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102329    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-cnibin\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102347    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102366    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-sys-fs\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102384    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-systemd-units\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102399    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-systemd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102412    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-ovn\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102428    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-script-lib\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.102612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102443    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-lib-modules\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102458    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-etc-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102473    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-socket-dir-parent\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102488    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-k8s-cni-cncf-io\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102504    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-netns\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102520    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-kubelet\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102545    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-multus-daemon-config\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102564    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbjqm\" (UniqueName: \"kubernetes.io/projected/651589de-cd01-4189-891b-7c30cca28409-kube-api-access-pbjqm\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102565    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102639    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-kubelet\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102688    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vk6\" (UniqueName: \"kubernetes.io/projected/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-kube-api-access-n7vk6\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102714    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-run\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102738    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-var-lib-kubelet\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102764    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-netns\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102786    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-node-log\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102810    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-socket-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102833    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-tmp\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102856    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-os-release\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102878    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-hostroot\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102909    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-host\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102928    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102952    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-kubernetes\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102973    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.102988    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9552277-2e3a-421e-a0d1-4a3131769f8c-tmp-dir\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103019    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlvrn\" (UniqueName: \"kubernetes.io/projected/d9552277-2e3a-421e-a0d1-4a3131769f8c-kube-api-access-qlvrn\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103050    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqpx\" (UniqueName: \"kubernetes.io/projected/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kube-api-access-nzqpx\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103073    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103603 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103095    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-tuned\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkw2n\" (UniqueName: \"kubernetes.io/projected/8b07ccbe-7598-45cd-b083-0a9e8b857aad-kube-api-access-vkw2n\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103132    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-var-lib-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103147    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztntg\" (UniqueName: \"kubernetes.io/projected/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-kube-api-access-ztntg\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103163    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103193    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-registration-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103210    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-modprobe-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103224    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-systemd\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103239    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9552277-2e3a-421e-a0d1-4a3131769f8c-hosts-file\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103261    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103313    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-slash\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-bin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.103999 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.103344    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-multus-certs\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.105087 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.105069    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h2fkn\""
Jul 16 21:50:18.105159 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.105100    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 21:50:18.105211 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.105167    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 21:50:18.106133 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.106113    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:50:18.140539 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.140521    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xmmkc"
Jul 16 21:50:18.149175 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.149154    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xmmkc"
Jul 16 21:50:18.193842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.193820    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 21:50:18.203598 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203553    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vk6\" (UniqueName: \"kubernetes.io/projected/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-kube-api-access-n7vk6\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.203598 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203590    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-run\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203617    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-var-lib-kubelet\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203643    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-netns\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203666    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-node-log\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203693    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203718    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-socket-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.203746 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203733    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-run\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-var-lib-kubelet\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203765    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-tmp\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203777    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-netns\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203799    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-os-release\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203834    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-socket-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203849    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-node-log\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203909    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-hostroot\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203958    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-os-release\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203976    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-hostroot\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.203973    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bae704ef-b67d-40d4-ae3f-b5bd4800861a-agent-certs\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204013    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bae704ef-b67d-40d4-ae3f-b5bd4800861a-konnectivity-ca\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204031    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-host\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204050    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.204065 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204075    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-host\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204111    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-kubernetes\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204126    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204170    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-kubernetes\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204203    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bba301eb-2534-4f99-be73-7a052e54d48a-iptables-alerter-script\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204233    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204259    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9552277-2e3a-421e-a0d1-4a3131769f8c-tmp-dir\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204298    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlvrn\" (UniqueName: \"kubernetes.io/projected/d9552277-2e3a-421e-a0d1-4a3131769f8c-kube-api-access-qlvrn\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204323    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-dbus\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204350    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqpx\" (UniqueName: \"kubernetes.io/projected/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kube-api-access-nzqpx\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204379    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204404    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-tuned\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204450    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkw2n\" (UniqueName: \"kubernetes.io/projected/8b07ccbe-7598-45cd-b083-0a9e8b857aad-kube-api-access-vkw2n\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204476    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-var-lib-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.204873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztntg\" (UniqueName: \"kubernetes.io/projected/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-kube-api-access-ztntg\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204532    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204557    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-registration-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204606    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-modprobe-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204633    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-systemd\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204684    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9552277-2e3a-421e-a0d1-4a3131769f8c-hosts-file\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204721    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204744    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9552277-2e3a-421e-a0d1-4a3131769f8c-hosts-file\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204750    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-slash\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204788    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-slash\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204783    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-bin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204843    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-multus-certs\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204874    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-log-socket\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204903    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-serviceca\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.205685 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204929    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204954    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204679    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.204997    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mvgg\" (UniqueName: \"kubernetes.io/projected/bba301eb-2534-4f99-be73-7a052e54d48a-kube-api-access-5mvgg\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205025    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.205059    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205100    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-modprobe-d\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.205113    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:18.70509774 +0000 UTC m=+2.007270417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205266    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-systemd\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205059    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhlzv\" (UniqueName: \"kubernetes.io/projected/e027d183-8aec-4901-b982-4b2c88a803a8-kube-api-access-jhlzv\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205337    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205364    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-sys\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205389    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-host\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205414    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-cni-binary-copy\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.208046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205440    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-etc-kubernetes\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205466    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-system-cni-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205462    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-var-lib-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205488    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205508    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-env-overrides\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205534    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bba301eb-2534-4f99-be73-7a052e54d48a-host-slash\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205537    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-sys\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205562    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-device-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205570    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205579    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9552277-2e3a-421e-a0d1-4a3131769f8c-tmp-dir\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205590    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-host\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205628    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-system-cni-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205631    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-registration-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205589    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj59b\" (UniqueName: \"kubernetes.io/projected/5351fd34-a13a-4dc7-84a1-f68100b8c43c-kube-api-access-kj59b\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.208616 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205806    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-config\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205835    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovn-node-metrics-cert\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205863    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-os-release\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205877    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205895    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205923    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-system-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205933    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-multus-certs\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205943    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205948    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-multus\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205982    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-bin\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205991    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-multus\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205997    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-log-socket\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206009    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-conf\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206035    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.209013 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206063    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysconfig\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206078    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-env-overrides\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206090    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-cnibin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.205809    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-etc-kubernetes\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206168    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-cnibin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206197    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-conf-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206224    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-netd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206251    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-cnibin\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206280    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206306    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-sys-fs\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206353    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-systemd-units\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206376    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-systemd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206396    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-ovn\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206422    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-script-lib\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-lib-modules\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.209424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206495    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-etc-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206520    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-socket-dir-parent\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206534    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-serviceca\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206549    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-k8s-cni-cncf-io\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206562    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-config\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206571    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-cni-binary-copy\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206575    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-netns\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206573    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-cnibin\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206593    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-systemd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206612    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-sys-fs\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-kubelet\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206642    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-system-cni-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-kubelet\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206638    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-run-ovn\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206667    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysctl-conf\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.209881 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206652    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-multus-daemon-config\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206687    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-bin\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206692    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-systemd-units\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206703    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206713    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbjqm\" (UniqueName: \"kubernetes.io/projected/651589de-cd01-4189-891b-7c30cca28409-kube-api-access-pbjqm\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206717    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206753    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206760    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-etc-openvswitch\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206765    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-k8s-cni-cncf-io\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206782    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-kubelet\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206789    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-lib-modules\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206810    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-kubelet-config\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206820    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e027d183-8aec-4901-b982-4b2c88a803a8-os-release\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206865    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-socket-dir-parent\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206885    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206903    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-multus-conf-dir\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206926    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-kubelet\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206949    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-sysconfig\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206996    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-device-dir\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.206999    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-run-netns\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207047    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-host-cni-netd\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207233    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovnkube-script-lib\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207233    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207288    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651589de-cd01-4189-891b-7c30cca28409-multus-daemon-config\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207303    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651589de-cd01-4189-891b-7c30cca28409-host-var-lib-cni-bin\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207310    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e027d183-8aec-4901-b982-4b2c88a803a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.207915    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-tmp\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.208991    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-ovn-node-metrics-cert\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.210718 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.209022    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b07ccbe-7598-45cd-b083-0a9e8b857aad-etc-tuned\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.215320 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.215300    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vk6\" (UniqueName: \"kubernetes.io/projected/b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91-kube-api-access-n7vk6\") pod \"node-ca-nx9cb\" (UID: \"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91\") " pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.221082 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.220804    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:18.221082 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.220833    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:18.221082 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.220847    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:18.221082 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.220925    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:18.720905124 +0000 UTC m=+2.023077789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:18.221082 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.221007    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbjqm\" (UniqueName: \"kubernetes.io/projected/651589de-cd01-4189-891b-7c30cca28409-kube-api-access-pbjqm\") pod \"multus-6ppht\" (UID: \"651589de-cd01-4189-891b-7c30cca28409\") " pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.221441 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.221179    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj59b\" (UniqueName: \"kubernetes.io/projected/5351fd34-a13a-4dc7-84a1-f68100b8c43c-kube-api-access-kj59b\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.221892 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.221871    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkw2n\" (UniqueName: \"kubernetes.io/projected/8b07ccbe-7598-45cd-b083-0a9e8b857aad-kube-api-access-vkw2n\") pod \"tuned-6zfwt\" (UID: \"8b07ccbe-7598-45cd-b083-0a9e8b857aad\") " pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.222023 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.221994    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztntg\" (UniqueName: \"kubernetes.io/projected/412fd8b9-d77f-467a-9eb8-7b64e753c9c9-kube-api-access-ztntg\") pod \"ovnkube-node-t77lx\" (UID: \"412fd8b9-d77f-467a-9eb8-7b64e753c9c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.222111 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.222077    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqpx\" (UniqueName: \"kubernetes.io/projected/53ac237d-3c99-4cd0-b39c-4c8912aeaaa1-kube-api-access-nzqpx\") pod \"aws-ebs-csi-driver-node-4mprw\" (UID: \"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.222312 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.222294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhlzv\" (UniqueName: \"kubernetes.io/projected/e027d183-8aec-4901-b982-4b2c88a803a8-kube-api-access-jhlzv\") pod \"multus-additional-cni-plugins-dr7m8\" (UID: \"e027d183-8aec-4901-b982-4b2c88a803a8\") " pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.222665 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.222649    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlvrn\" (UniqueName: \"kubernetes.io/projected/d9552277-2e3a-421e-a0d1-4a3131769f8c-kube-api-access-qlvrn\") pod \"node-resolver-n9tkf\" (UID: \"d9552277-2e3a-421e-a0d1-4a3131769f8c\") " pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-kubelet-config\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307388    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307426    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bae704ef-b67d-40d4-ae3f-b5bd4800861a-agent-certs\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307459    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bae704ef-b67d-40d4-ae3f-b5bd4800861a-konnectivity-ca\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307486    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bba301eb-2534-4f99-be73-7a052e54d48a-iptables-alerter-script\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307539    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-dbus\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307595    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mvgg\" (UniqueName: \"kubernetes.io/projected/bba301eb-2534-4f99-be73-7a052e54d48a-kube-api-access-5mvgg\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307653    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bba301eb-2534-4f99-be73-7a052e54d48a-host-slash\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.307737    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bba301eb-2534-4f99-be73-7a052e54d48a-host-slash\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.308273    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bae704ef-b67d-40d4-ae3f-b5bd4800861a-konnectivity-ca\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.308780    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bba301eb-2534-4f99-be73-7a052e54d48a-iptables-alerter-script\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.309209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.308865    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-dbus\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.309892 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.309282    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ab4a654-98ee-45e0-9284-62fe8e871a27-kubelet-config\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.309892 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.309320    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:18.309892 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.309489    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:18.809454457 +0000 UTC m=+2.111627126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:18.311357 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.311326    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bae704ef-b67d-40d4-ae3f-b5bd4800861a-agent-certs\") pod \"konnectivity-agent-kvhc6\" (UID: \"bae704ef-b67d-40d4-ae3f-b5bd4800861a\") " pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.318003 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.317982    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mvgg\" (UniqueName: \"kubernetes.io/projected/bba301eb-2534-4f99-be73-7a052e54d48a-kube-api-access-5mvgg\") pod \"iptables-alerter-487vh\" (UID: \"bba301eb-2534-4f99-be73-7a052e54d48a\") " pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.407932 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.407898    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nx9cb"
Jul 16 21:50:18.424749 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.424730    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dr7m8"
Jul 16 21:50:18.426941 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.426907    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74669e993430a0a3567c9a05c1f57949.slice/crio-6a5cc12d90917985dd7014e77c4498b02705de6bf78048fe3f0c9159e0ca6488 WatchSource:0}: Error finding container 6a5cc12d90917985dd7014e77c4498b02705de6bf78048fe3f0c9159e0ca6488: Status 404 returned error can't find the container with id 6a5cc12d90917985dd7014e77c4498b02705de6bf78048fe3f0c9159e0ca6488
Jul 16 21:50:18.427138 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.427123    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb88a8a6b_c3f2_4b14_a7f2_b6e97c083f91.slice/crio-7ab1aab2a05ebb6a4d5e640c90803cffe73e12f9be2f42e26e9039ea26fdd09f WatchSource:0}: Error finding container 7ab1aab2a05ebb6a4d5e640c90803cffe73e12f9be2f42e26e9039ea26fdd09f: Status 404 returned error can't find the container with id 7ab1aab2a05ebb6a4d5e640c90803cffe73e12f9be2f42e26e9039ea26fdd09f
Jul 16 21:50:18.433984 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.433966    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:50:18.438473 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.438444    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode027d183_8aec_4901_b982_4b2c88a803a8.slice/crio-1ad59ce7034bfc6885c2278e93f17785dd0a85b4efc9ebc2218bf568e83f64ef WatchSource:0}: Error finding container 1ad59ce7034bfc6885c2278e93f17785dd0a85b4efc9ebc2218bf568e83f64ef: Status 404 returned error can't find the container with id 1ad59ce7034bfc6885c2278e93f17785dd0a85b4efc9ebc2218bf568e83f64ef
Jul 16 21:50:18.439336 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.439293    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1e5172fa4dc5ea0b1b27edef40bb69.slice/crio-8d82a351e7769d1022907e2b4444cb0be41c5495a1a53df4859b0467ace5397f WatchSource:0}: Error finding container 8d82a351e7769d1022907e2b4444cb0be41c5495a1a53df4859b0467ace5397f: Status 404 returned error can't find the container with id 8d82a351e7769d1022907e2b4444cb0be41c5495a1a53df4859b0467ace5397f
Jul 16 21:50:18.442747 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.442725    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:18.453885 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.453829    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412fd8b9_d77f_467a_9eb8_7b64e753c9c9.slice/crio-723a0cd66dcb780d3f137e265c4d0056810c9c770572f943f684cb86823c9297 WatchSource:0}: Error finding container 723a0cd66dcb780d3f137e265c4d0056810c9c770572f943f684cb86823c9297: Status 404 returned error can't find the container with id 723a0cd66dcb780d3f137e265c4d0056810c9c770572f943f684cb86823c9297
Jul 16 21:50:18.459320 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.459301    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw"
Jul 16 21:50:18.465458 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.465437    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zfwt"
Jul 16 21:50:18.467540 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.467521    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ac237d_3c99_4cd0_b39c_4c8912aeaaa1.slice/crio-3ac9fa35697bbf4385ad815595cbc5b1ff9d0239978064d4efe65aeea18dd25f WatchSource:0}: Error finding container 3ac9fa35697bbf4385ad815595cbc5b1ff9d0239978064d4efe65aeea18dd25f: Status 404 returned error can't find the container with id 3ac9fa35697bbf4385ad815595cbc5b1ff9d0239978064d4efe65aeea18dd25f
Jul 16 21:50:18.472880 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.472857    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b07ccbe_7598_45cd_b083_0a9e8b857aad.slice/crio-cc00b408a0a1faaa9d659313a60205bed74ffded5cc0ece1a0d1ae85b89441b8 WatchSource:0}: Error finding container cc00b408a0a1faaa9d659313a60205bed74ffded5cc0ece1a0d1ae85b89441b8: Status 404 returned error can't find the container with id cc00b408a0a1faaa9d659313a60205bed74ffded5cc0ece1a0d1ae85b89441b8
Jul 16 21:50:18.480490 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.480473    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n9tkf"
Jul 16 21:50:18.489012 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.488994    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9552277_2e3a_421e_a0d1_4a3131769f8c.slice/crio-92ba4e139286417e750bacdb7d542d95f67cd6c04251eceafe2a4856e5ac6857 WatchSource:0}: Error finding container 92ba4e139286417e750bacdb7d542d95f67cd6c04251eceafe2a4856e5ac6857: Status 404 returned error can't find the container with id 92ba4e139286417e750bacdb7d542d95f67cd6c04251eceafe2a4856e5ac6857
Jul 16 21:50:18.500541 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.500521    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6ppht"
Jul 16 21:50:18.508687 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.508665    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-487vh"
Jul 16 21:50:18.509205 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.509185    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651589de_cd01_4189_891b_7c30cca28409.slice/crio-cb03096181de4d799151c15c8259a66e7311abbde8ed1d92022b50b3cc09ea00 WatchSource:0}: Error finding container cb03096181de4d799151c15c8259a66e7311abbde8ed1d92022b50b3cc09ea00: Status 404 returned error can't find the container with id cb03096181de4d799151c15c8259a66e7311abbde8ed1d92022b50b3cc09ea00
Jul 16 21:50:18.516737 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.516716    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba301eb_2534_4f99_be73_7a052e54d48a.slice/crio-933ffecafebaf32e36266d1160a02c0fea8712904b292418c4abbca635ee0eab WatchSource:0}: Error finding container 933ffecafebaf32e36266d1160a02c0fea8712904b292418c4abbca635ee0eab: Status 404 returned error can't find the container with id 933ffecafebaf32e36266d1160a02c0fea8712904b292418c4abbca635ee0eab
Jul 16 21:50:18.532246 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.532229    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:18.541484 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:18.541460    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae704ef_b67d_40d4_ae3f_b5bd4800861a.slice/crio-259ae1d93cbd4b004b72ce51b5e4e61458545d2596577dabccd275639ec1ce38 WatchSource:0}: Error finding container 259ae1d93cbd4b004b72ce51b5e4e61458545d2596577dabccd275639ec1ce38: Status 404 returned error can't find the container with id 259ae1d93cbd4b004b72ce51b5e4e61458545d2596577dabccd275639ec1ce38
Jul 16 21:50:18.542702 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.542681    2551 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-kvhc6_kube-system(bae704ef-b67d-40d4-ae3f-b5bd4800861a): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 16 21:50:18.542781 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.542711    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="kube-system/konnectivity-agent-kvhc6" podUID="bae704ef-b67d-40d4-ae3f-b5bd4800861a"
Jul 16 21:50:18.710401 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.710280    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:18.710542 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.710426    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:18.710542 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.710494    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:19.710477576 +0000 UTC m=+3.012650229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:18.811084 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.811028    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:18.811313 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.811209    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:18.811313 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811238    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:18.811313 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811282    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:18.811313 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811297    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:18.811546 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811332    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:18.811546 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811366    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:19.811346276 +0000 UTC m=+3.113518950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:18.811546 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:18.811389    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:19.811377952 +0000 UTC m=+3.113550605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:18.906298 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:18.905922    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:19.085306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.085220    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:19.092339 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.092294    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:50:19.152725 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.152643    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:45:18 +0000 UTC" deadline="2028-05-02 19:42:43.141228858 +0000 UTC"
Jul 16 21:50:19.152725 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.152723    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15741h52m23.988510524s"
Jul 16 21:50:19.325961 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.325900    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kvhc6" event={"ID":"bae704ef-b67d-40d4-ae3f-b5bd4800861a","Type":"ContainerStarted","Data":"259ae1d93cbd4b004b72ce51b5e4e61458545d2596577dabccd275639ec1ce38"}
Jul 16 21:50:19.331759 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.331725    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9e6cee53a67cbe2094b527990bd1d0d668c31dc268b25bb101309f07fab93715\\\": ErrImagePull: pull QPS exceeded\"" pod="kube-system/konnectivity-agent-kvhc6" podUID="bae704ef-b67d-40d4-ae3f-b5bd4800861a"
Jul 16 21:50:19.352778 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.352690    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n9tkf" event={"ID":"d9552277-2e3a-421e-a0d1-4a3131769f8c","Type":"ContainerStarted","Data":"92ba4e139286417e750bacdb7d542d95f67cd6c04251eceafe2a4856e5ac6857"}
Jul 16 21:50:19.393905 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.393864    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zfwt" event={"ID":"8b07ccbe-7598-45cd-b083-0a9e8b857aad","Type":"ContainerStarted","Data":"cc00b408a0a1faaa9d659313a60205bed74ffded5cc0ece1a0d1ae85b89441b8"}
Jul 16 21:50:19.427909 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.427867    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw" event={"ID":"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1","Type":"ContainerStarted","Data":"3ac9fa35697bbf4385ad815595cbc5b1ff9d0239978064d4efe65aeea18dd25f"}
Jul 16 21:50:19.448184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.448132    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal" event={"ID":"fb1e5172fa4dc5ea0b1b27edef40bb69","Type":"ContainerStarted","Data":"8d82a351e7769d1022907e2b4444cb0be41c5495a1a53df4859b0467ace5397f"}
Jul 16 21:50:19.462963 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.462917    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerStarted","Data":"1ad59ce7034bfc6885c2278e93f17785dd0a85b4efc9ebc2218bf568e83f64ef"}
Jul 16 21:50:19.483231 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.483186    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nx9cb" event={"ID":"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91","Type":"ContainerStarted","Data":"7ab1aab2a05ebb6a4d5e640c90803cffe73e12f9be2f42e26e9039ea26fdd09f"}
Jul 16 21:50:19.491399 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.491365    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-487vh" event={"ID":"bba301eb-2534-4f99-be73-7a052e54d48a","Type":"ContainerStarted","Data":"933ffecafebaf32e36266d1160a02c0fea8712904b292418c4abbca635ee0eab"}
Jul 16 21:50:19.497649 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.497617    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6ppht" event={"ID":"651589de-cd01-4189-891b-7c30cca28409","Type":"ContainerStarted","Data":"cb03096181de4d799151c15c8259a66e7311abbde8ed1d92022b50b3cc09ea00"}
Jul 16 21:50:19.508233 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.508203    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"723a0cd66dcb780d3f137e265c4d0056810c9c770572f943f684cb86823c9297"}
Jul 16 21:50:19.518626 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.518565    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal" event={"ID":"74669e993430a0a3567c9a05c1f57949","Type":"ContainerStarted","Data":"6a5cc12d90917985dd7014e77c4498b02705de6bf78048fe3f0c9159e0ca6488"}
Jul 16 21:50:19.721086 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.720919    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:19.721086 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.721074    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:19.721348 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.721144    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:21.721122823 +0000 UTC m=+5.023295503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:19.822314 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.822252    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:19.822531 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:19.822337    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:19.822531 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822473    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:19.822531 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822507    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:19.822531 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822524    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:19.822751 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822540    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:19.822751 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822553    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:21.822531218 +0000 UTC m=+5.124703872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:19.822751 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:19.822591    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:21.822574395 +0000 UTC m=+5.124747049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:20.153827 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:20.153707    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:45:18 +0000 UTC" deadline="2028-02-21 18:47:07.96900287 +0000 UTC"
Jul 16 21:50:20.153827 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:20.153775    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14036h56m47.815232077s"
Jul 16 21:50:20.290646 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:20.290607    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:20.290837 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:20.290746    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:20.291198 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:20.291173    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:20.291307 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:20.291278    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:20.291414 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:20.291380    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:20.291501 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:20.291480    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:21.741709 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:21.741670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:21.742206 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.741864    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:21.742206 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.741974    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:25.741949655 +0000 UTC m=+9.044122338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:21.843077 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:21.842987    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:21.843077 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:21.843051    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843167    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843237    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843247    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:25.843226415 +0000 UTC m=+9.145399095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843259    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843275    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:21.843347 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:21.843342    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:25.843324681 +0000 UTC m=+9.145497339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:22.289654 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:22.289568    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:22.289843 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:22.289718    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:22.290112 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:22.290094    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:22.290213 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:22.290193    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:22.290297 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:22.290247    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:22.290371 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:22.290351    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:24.290426    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:24.290464    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:24.290435    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:24.290574    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:24.290688    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:24.290997 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:24.290768    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:25.779108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:25.778466    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:25.779108 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.778645    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:25.779108 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.778712    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:33.778691952 +0000 UTC m=+17.080864608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:25.879415 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:25.879373    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:25.879618 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:25.879436    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:25.879618 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879612    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:25.879745 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879633    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:25.879745 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879647    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:25.879745 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879711    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:33.879688822 +0000 UTC m=+17.181861478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:25.879909 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879783    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:25.879909 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:25.879816    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:33.879805293 +0000 UTC m=+17.181977948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:26.290034 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:26.289991    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:26.290236 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:26.290042    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:26.290236 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:26.290000    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:26.290236 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:26.290140    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:26.290236 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:26.290204    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:26.290512 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:26.290318    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:28.290355 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:28.290323    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:28.290842 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:28.290456    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:28.290842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:28.290514    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:28.290842 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:28.290652    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:28.291030 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:28.290885    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:28.291030 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:28.290998    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:30.290323 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:30.290280    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:30.290754 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:30.290280    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:30.290754 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:30.290410    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:30.290754 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:30.290280    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:30.290754 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:30.290528    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:30.290754 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:30.290632    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:32.290325 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:32.290285    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:32.290839 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:32.290285    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:32.290839 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:32.290426    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:32.290839 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:32.290520    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:32.290839 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:32.290285    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:32.290839 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:32.290625    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:33.842790 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:33.842747    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:33.843237 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.842940    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:33.843237 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.843023    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:50:49.843001407 +0000 UTC m=+33.145174063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:33.943652 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:33.943613    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:33.943837 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:33.943673    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:33.943837 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943799    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:33.943961 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943866    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:33.943961 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943880    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:49.943859633 +0000 UTC m=+33.246032295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:33.943961 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943891    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:33.943961 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943903    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:33.943961 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:33.943941    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:50:49.943930325 +0000 UTC m=+33.246102976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:34.290591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:34.290472    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:34.290748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:34.290595    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:34.290748 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:34.290600    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:34.290748 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:34.290701    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:34.290748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:34.290603    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:34.290947 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:34.290793    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:36.290366 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:36.290326    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:36.290366 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:36.290357    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:36.290912 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:36.290435    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:36.290912 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:36.290449    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:36.290912 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:36.290539    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:36.290912 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:36.290631    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:38.290164 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:38.290120    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:38.290164 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:38.290145    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:38.290650 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:38.290139    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:38.290650 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:38.290275    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:38.290650 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:38.290374    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:38.290650 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:38.290459    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:39.569842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.569634    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n9tkf" event={"ID":"d9552277-2e3a-421e-a0d1-4a3131769f8c","Type":"ContainerStarted","Data":"c27d7666114c67652765a08405fb32d386b1f0c2c47b25ef46675d29a75d98b0"}
Jul 16 21:50:39.571412 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.571380    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zfwt" event={"ID":"8b07ccbe-7598-45cd-b083-0a9e8b857aad","Type":"ContainerStarted","Data":"a9e54feccf2ba1012170f7bf5c27f7bf3260698792628f7727fb14b2457031a6"}
Jul 16 21:50:39.573212 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.573152    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw" event={"ID":"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1","Type":"ContainerStarted","Data":"437a0ee755a8eba0a652463ac5f47b650014537a0f6a532a58275bdce532a68d"}
Jul 16 21:50:39.574745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.574712    2551 generic.go:356] "Generic (PLEG): container finished" podID="fb1e5172fa4dc5ea0b1b27edef40bb69" containerID="6c36d69ce5cef24b16348e60d3fd539e21e4b0832a1de2c1c36ddef9de611ba6" exitCode=0
Jul 16 21:50:39.574838 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.574782    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal" event={"ID":"fb1e5172fa4dc5ea0b1b27edef40bb69","Type":"ContainerDied","Data":"6c36d69ce5cef24b16348e60d3fd539e21e4b0832a1de2c1c36ddef9de611ba6"}
Jul 16 21:50:39.576779 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.576748    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerStarted","Data":"d2405877c4bb755a02d8ab66f3b8132c6d8c2c2f656fffd793903db92082d3dc"}
Jul 16 21:50:39.578612 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.578575    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nx9cb" event={"ID":"b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91","Type":"ContainerStarted","Data":"c815e2205204ddf27f94c7752480c04000fa0c0d0a1ebb2a7d27549a57c1e0de"}
Jul 16 21:50:39.580302 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.580277    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6ppht" event={"ID":"651589de-cd01-4189-891b-7c30cca28409","Type":"ContainerStarted","Data":"02184dae25219c150a6bae58a9997dbb2abca70f7c51303062d6497671cca833"}
Jul 16 21:50:39.582495 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.582471    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"774665a8e2f149ef38d8bc65cb59aefa26a0d6b84420df702663f089c77360da"}
Jul 16 21:50:39.582595 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.582542    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"b24ca9a97daea1904ca60b927e200d24c507440a6af8289f1ed8b1b1beb9ad7b"}
Jul 16 21:50:39.582595 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.582578    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"15d15daedd64454cfd569133da71caa4b1d3c5fa26b22cf25e70efc7950d4dab"}
Jul 16 21:50:39.583874 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.583847    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal" event={"ID":"74669e993430a0a3567c9a05c1f57949","Type":"ContainerStarted","Data":"cf592cc89330746d316fd8291b757244a566e6bcdd81a2866c1dc9b3975b50eb"}
Jul 16 21:50:39.619977 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.619934    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n9tkf" podStartSLOduration=1.06078821 podStartE2EDuration="21.619913519s" totalImagesPullingTime="20.559125309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:39.589867361 +0000 UTC m=+22.892040035" watchObservedRunningTime="2026-07-16 21:50:39.619913519 +0000 UTC m=+22.922086393"
Jul 16 21:50:39.651599 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.651542    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6zfwt" podStartSLOduration=1.02021757 podStartE2EDuration="21.651524761s" totalImagesPullingTime="20.631307191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:39.620029532 +0000 UTC m=+22.922202197" watchObservedRunningTime="2026-07-16 21:50:39.651524761 +0000 UTC m=+22.953697434"
Jul 16 21:50:39.677210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.677170    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6ppht" podStartSLOduration=1.069108127 podStartE2EDuration="21.677154122s" totalImagesPullingTime="20.608045995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:39.677142569 +0000 UTC m=+22.979315241" watchObservedRunningTime="2026-07-16 21:50:39.677154122 +0000 UTC m=+22.979326794"
Jul 16 21:50:39.703076 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.703034    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nx9cb" podStartSLOduration=1.087016471 podStartE2EDuration="21.70301986s" totalImagesPullingTime="20.616003389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:39.702815285 +0000 UTC m=+23.004987958" watchObservedRunningTime="2026-07-16 21:50:39.70301986 +0000 UTC m=+23.005192533"
Jul 16 21:50:39.728730 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:39.728687    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-41.ec2.internal" podStartSLOduration=21.728670442 podStartE2EDuration="21.728670442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:39.728479919 +0000 UTC m=+23.030652593" watchObservedRunningTime="2026-07-16 21:50:39.728670442 +0000 UTC m=+23.030843114"
Jul 16 21:50:40.290203 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.290026    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:40.290325 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.290072    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:40.290325 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:40.290291    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:40.290415 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.290102    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:40.290415 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:40.290350    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:40.290487 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:40.290423    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:40.586731 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.586699    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="d2405877c4bb755a02d8ab66f3b8132c6d8c2c2f656fffd793903db92082d3dc" exitCode=0
Jul 16 21:50:40.587117 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.586770    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"d2405877c4bb755a02d8ab66f3b8132c6d8c2c2f656fffd793903db92082d3dc"}
Jul 16 21:50:40.588190 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.588161    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-487vh" event={"ID":"bba301eb-2534-4f99-be73-7a052e54d48a","Type":"ContainerStarted","Data":"b33afca1c4041af15984fb2f1972b1358f664a8d9a019fec39ebdcec614ee433"}
Jul 16 21:50:40.591067 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.591045    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"63d2f8d24756379de84425f655410453176412c515c0e41a0e980f0e4893ffef"}
Jul 16 21:50:40.591173 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.591104    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"e79bddb4dfa50f0239abee67d04568ded5bc99faeee225f089fb05ad4f66d311"}
Jul 16 21:50:40.591173 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.591148    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"1bb157752987c9abb1e993be9241ffba281f88922202730ecc7d22b17c176729"}
Jul 16 21:50:40.592443 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.592419    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kvhc6" event={"ID":"bae704ef-b67d-40d4-ae3f-b5bd4800861a","Type":"ContainerStarted","Data":"3fe0bfd6e031babec70226fec7322b7174f94e217541b0f7c089c38f3b096509"}
Jul 16 21:50:40.656786 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.656729    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-487vh" podStartSLOduration=2.069915688 podStartE2EDuration="22.656713534s" totalImagesPullingTime="20.586797846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:40.655100714 +0000 UTC m=+23.957273387" watchObservedRunningTime="2026-07-16 21:50:40.656713534 +0000 UTC m=+23.958886206"
Jul 16 21:50:40.683353 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:40.683300    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kvhc6" podStartSLOduration=0.76623752 podStartE2EDuration="22.683283274s" totalImagesPullingTime="21.917045754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-16 21:50:40.683227652 +0000 UTC m=+23.985400327" watchObservedRunningTime="2026-07-16 21:50:40.683283274 +0000 UTC m=+23.985455947"
Jul 16 21:50:41.228997 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:41.228973    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 21:50:41.596904 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:41.596870    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw" event={"ID":"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1","Type":"ContainerStarted","Data":"32cd702fa898bf9f626038db3cf97fe059e16c46c663e2a039542035e09530aa"}
Jul 16 21:50:41.598950 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:41.598917    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal" event={"ID":"fb1e5172fa4dc5ea0b1b27edef40bb69","Type":"ContainerStarted","Data":"b463a183e7f0ea9d3d7b35038bc6f318bf9c039e0d968356d8e542ac79f55b6b"}
Jul 16 21:50:41.618399 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:41.618342    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-41.ec2.internal" podStartSLOduration=23.618322573 podStartE2EDuration="23.618322573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:41.617502093 +0000 UTC m=+24.919674766" watchObservedRunningTime="2026-07-16 21:50:41.618322573 +0000 UTC m=+24.920495248"
Jul 16 21:50:42.194973 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.194843    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:50:41.22899162Z","UUID":"3a45751c-da7d-49eb-bb1e-edd82db27f91","Handler":null,"Name":"","Endpoint":""}
Jul 16 21:50:42.196605 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.196582    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 21:50:42.196716 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.196627    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 21:50:42.290677 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.290636    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:42.290844 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.290646    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:42.290844 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:42.290767    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:42.290949 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:42.290863    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:42.290949 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.290655    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:42.291051 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:42.290959    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:42.397483 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.397441    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:42.398025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.398002    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:42.601943 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.601918    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:42.602507 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:42.602417    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kvhc6"
Jul 16 21:50:43.605808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:43.605770    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw" event={"ID":"53ac237d-3c99-4cd0-b39c-4c8912aeaaa1","Type":"ContainerStarted","Data":"2a7b3e134f1f770efe63a30d70effa9dee3c804c4e3e636b9e9b4f874a3b3a05"}
Jul 16 21:50:43.609122 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:43.609095    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"3cb1ebe7a69cde35d27dcde7e0564a29cc724c80ee5d2e78f334a9c0fc8df954"}
Jul 16 21:50:43.630283 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:43.630220    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mprw" podStartSLOduration=2.122377136 podStartE2EDuration="25.630201223s" totalImagesPullingTime="23.507824087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:43.630170066 +0000 UTC m=+26.932342739" watchObservedRunningTime="2026-07-16 21:50:43.630201223 +0000 UTC m=+26.932373895"
Jul 16 21:50:44.290262 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:44.290160    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:44.290262 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:44.290183    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:44.290511 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:44.290315    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:44.290511 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:44.290316    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:44.290511 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:44.290434    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:44.290673 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:44.290515    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:45.615962 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.615750    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="830384ba93d61aaca677f3b5b61eead24d3284054b98eae056e04a6039a2459a" exitCode=0
Jul 16 21:50:45.615962 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.615810    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"830384ba93d61aaca677f3b5b61eead24d3284054b98eae056e04a6039a2459a"}
Jul 16 21:50:45.619697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.619673    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" event={"ID":"412fd8b9-d77f-467a-9eb8-7b64e753c9c9","Type":"ContainerStarted","Data":"a3ce2f6d2bcfd47cbd8dbc308e93564e38b3551b714ef5961a61946291e82ae1"}
Jul 16 21:50:45.619923 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.619909    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:45.635566 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.635543    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:45.679586 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:45.679535    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx" podStartSLOduration=6.9417665379999995 podStartE2EDuration="27.679519362s" totalImagesPullingTime="20.737752824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:45.679069994 +0000 UTC m=+28.981242667" watchObservedRunningTime="2026-07-16 21:50:45.679519362 +0000 UTC m=+28.981692034"
Jul 16 21:50:46.289763 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.289732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:46.289965 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.289775    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:46.289965 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.289742    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:46.289965 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:46.289847    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:46.289965 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:46.289902    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:46.290230 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:46.289978    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:46.622513 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.622450    2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 16 21:50:46.622920 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.622864    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:46.637836 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.637806    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:46.682720 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:46.682685    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:50:47.477303 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.477069    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2m5v5"]
Jul 16 21:50:47.477480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.477421    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:47.477556 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:47.477535    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:47.477787 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.477764    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shstt"]
Jul 16 21:50:47.477980 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.477892    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:47.478033 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:47.477998    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:47.479289 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.479274    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9vjn"]
Jul 16 21:50:47.479354 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.479345    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:47.479429 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:47.479413    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:47.626701 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.626618    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="cce7fff095883241ba3efa013c5109481120ee3d8eae30d8029e7478eb077914" exitCode=0
Jul 16 21:50:47.627025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:47.626688    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"cce7fff095883241ba3efa013c5109481120ee3d8eae30d8029e7478eb077914"}
Jul 16 21:50:48.406654 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:48.406565    2551 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0dac675d2b2751ec266bb725c6ebd537c5318f2735f4e46cc0bd0916ceb94e94"
Jul 16 21:50:48.406823 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:48.406685    2551 kuberuntime_manager.go:1459] "Unhandled Error" err="init container routeoverride-cni start failed in pod multus-additional-cni-plugins-dr7m8_openshift-multus(e027d183-8aec-4901-b982-4b2c88a803a8): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 16 21:50:48.406823 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:48.406718    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"routeoverride-cni\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" podUID="e027d183-8aec-4901-b982-4b2c88a803a8"
Jul 16 21:50:48.629720 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:48.629693    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"routeoverride-cni\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0dac675d2b2751ec266bb725c6ebd537c5318f2735f4e46cc0bd0916ceb94e94\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" podUID="e027d183-8aec-4901-b982-4b2c88a803a8"
Jul 16 21:50:49.293421 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.293387    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:49.293575 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.293489    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:49.293575 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.293499    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:49.293575 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.293521    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:49.293674 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.293615    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:49.293704 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.293690    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:49.860199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.860167    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:49.860601 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.860297    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:49.860601 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.860351    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs podName:5351fd34-a13a-4dc7-84a1-f68100b8c43c nodeName:}" failed. No retries permitted until 2026-07-16 21:51:21.860336385 +0000 UTC m=+65.162509035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs") pod "network-metrics-daemon-2m5v5" (UID: "5351fd34-a13a-4dc7-84a1-f68100b8c43c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:50:49.961032 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.960997    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:49.961032 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:49.961036    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:49.961229 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961153    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:49.961229 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961167    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:50:49.961229 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961184    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:50:49.961229 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961197    2551 projected.go:196] Error preparing data for projected volume kube-api-access-g2sw8 for pod openshift-network-diagnostics/network-check-target-shstt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:49.961229 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961221    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret podName:2ab4a654-98ee-45e0-9284-62fe8e871a27 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:21.961201017 +0000 UTC m=+65.263373687 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret") pod "global-pull-secret-syncer-l9vjn" (UID: "2ab4a654-98ee-45e0-9284-62fe8e871a27") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:50:49.961409 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:49.961237    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8 podName:1e70a2b4-df90-4dfb-9a10-db3c438750dd nodeName:}" failed. No retries permitted until 2026-07-16 21:51:21.961229097 +0000 UTC m=+65.263401748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2sw8" (UniqueName: "kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8") pod "network-check-target-shstt" (UID: "1e70a2b4-df90-4dfb-9a10-db3c438750dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:50:51.290157 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:51.290121    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:51.290157 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:51.290140    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:51.290674 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:51.290123    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:51.290674 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:51.290228    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shstt" podUID="1e70a2b4-df90-4dfb-9a10-db3c438750dd"
Jul 16 21:50:51.290674 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:51.290320    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9vjn" podUID="2ab4a654-98ee-45e0-9284-62fe8e871a27"
Jul 16 21:50:51.290674 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:51.290400    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m5v5" podUID="5351fd34-a13a-4dc7-84a1-f68100b8c43c"
Jul 16 21:50:52.982935 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:52.982912    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-41.ec2.internal" event="NodeReady"
Jul 16 21:50:52.983424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:52.983043    2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 21:50:53.045342 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.045312    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-km54j"]
Jul 16 21:50:53.049653 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.049637    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.052199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.052147    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lxh7b"]
Jul 16 21:50:53.052452 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.052433    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 21:50:53.052542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.052467    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2kl8b\""
Jul 16 21:50:53.052542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.052439    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 21:50:53.056716 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.056374    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.061075 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.061058    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 21:50:53.061163 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.061144    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 21:50:53.061234 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.061160    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5hcvl\""
Jul 16 21:50:53.063581 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.063554    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-km54j"]
Jul 16 21:50:53.064735 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.064714    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 21:50:53.066818 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.066795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lxh7b"]
Jul 16 21:50:53.183538 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183505    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61b46972-876c-430c-b9de-187f8f9e9c81-config-volume\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.183538 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183541    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b46972-876c-430c-b9de-187f8f9e9c81-tmp-dir\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.183745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183569    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvf4r\" (UniqueName: \"kubernetes.io/projected/59cc2fbf-efaf-4b09-870a-13472c7113c9-kube-api-access-cvf4r\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.183745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183636    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.183745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183669    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnlrs\" (UniqueName: \"kubernetes.io/projected/61b46972-876c-430c-b9de-187f8f9e9c81-kube-api-access-rnlrs\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.183745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.183684    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59cc2fbf-efaf-4b09-870a-13472c7113c9-cert\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.284056 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.283979    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnlrs\" (UniqueName: \"kubernetes.io/projected/61b46972-876c-430c-b9de-187f8f9e9c81-kube-api-access-rnlrs\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.284056 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284012    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59cc2fbf-efaf-4b09-870a-13472c7113c9-cert\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.284056 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284053    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61b46972-876c-430c-b9de-187f8f9e9c81-config-volume\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.284314 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284071    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b46972-876c-430c-b9de-187f8f9e9c81-tmp-dir\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.284314 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284175    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvf4r\" (UniqueName: \"kubernetes.io/projected/59cc2fbf-efaf-4b09-870a-13472c7113c9-kube-api-access-cvf4r\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.284314 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284237    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.284462 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:53.284356    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:50:53.284462 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:53.284418    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls podName:61b46972-876c-430c-b9de-187f8f9e9c81 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:53.784398805 +0000 UTC m=+37.086571464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls") pod "dns-default-km54j" (UID: "61b46972-876c-430c-b9de-187f8f9e9c81") : secret "dns-default-metrics-tls" not found
Jul 16 21:50:53.284462 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284431    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b46972-876c-430c-b9de-187f8f9e9c81-tmp-dir\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.284717 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.284695    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61b46972-876c-430c-b9de-187f8f9e9c81-config-volume\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.288145 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.288122    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59cc2fbf-efaf-4b09-870a-13472c7113c9-cert\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.292980 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.292932    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnlrs\" (UniqueName: \"kubernetes.io/projected/61b46972-876c-430c-b9de-187f8f9e9c81-kube-api-access-rnlrs\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.293710 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.293690    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:50:53.293817 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.293688    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:50:53.294011 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.293690    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:50:53.294108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.294069    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvf4r\" (UniqueName: \"kubernetes.io/projected/59cc2fbf-efaf-4b09-870a-13472c7113c9-kube-api-access-cvf4r\") pod \"ingress-canary-lxh7b\" (UID: \"59cc2fbf-efaf-4b09-870a-13472c7113c9\") " pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.296949 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.296927    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:50:53.297094 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.296977    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:50:53.297094 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.297025    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:50:53.297357 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.297343    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmhws\""
Jul 16 21:50:53.298358 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.298339    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qgzlt\""
Jul 16 21:50:53.298452 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.298338    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:50:53.366120 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.366084    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lxh7b"
Jul 16 21:50:53.502428 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.502238    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lxh7b"]
Jul 16 21:50:53.505714 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:53.505686    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59cc2fbf_efaf_4b09_870a_13472c7113c9.slice/crio-1b2207ccf5cf915ede6000e930f2d7c60a2ff573204ecc42966c8f062fb401ce WatchSource:0}: Error finding container 1b2207ccf5cf915ede6000e930f2d7c60a2ff573204ecc42966c8f062fb401ce: Status 404 returned error can't find the container with id 1b2207ccf5cf915ede6000e930f2d7c60a2ff573204ecc42966c8f062fb401ce
Jul 16 21:50:53.639845 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.639760    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lxh7b" event={"ID":"59cc2fbf-efaf-4b09-870a-13472c7113c9","Type":"ContainerStarted","Data":"1b2207ccf5cf915ede6000e930f2d7c60a2ff573204ecc42966c8f062fb401ce"}
Jul 16 21:50:53.788439 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:53.788403    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:53.788577 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:53.788518    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:50:53.788577 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:53.788568    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls podName:61b46972-876c-430c-b9de-187f8f9e9c81 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:54.788554423 +0000 UTC m=+38.090727073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls") pod "dns-default-km54j" (UID: "61b46972-876c-430c-b9de-187f8f9e9c81") : secret "dns-default-metrics-tls" not found
Jul 16 21:50:54.796301 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:54.796247    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:54.796761 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:54.796370    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:50:54.796761 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:54.796441    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls podName:61b46972-876c-430c-b9de-187f8f9e9c81 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:56.796421587 +0000 UTC m=+40.098594241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls") pod "dns-default-km54j" (UID: "61b46972-876c-430c-b9de-187f8f9e9c81") : secret "dns-default-metrics-tls" not found
Jul 16 21:50:56.050892 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.050864    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-748d5b8ddb-97qq5"]
Jul 16 21:50:56.053759 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.053742    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.059707 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.059685    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 21:50:56.061046 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061025    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 21:50:56.061142 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061044    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 21:50:56.061205 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061146    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 21:50:56.061205 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061171    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 21:50:56.061473 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061446    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xlpld\""
Jul 16 21:50:56.061837 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.061822    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 21:50:56.078386 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.078361    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-748d5b8ddb-97qq5"]
Jul 16 21:50:56.190895 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.190863    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-5j647"]
Jul 16 21:50:56.193800 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.193782    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"]
Jul 16 21:50:56.193925 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.193910    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.196471 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.196447    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"]
Jul 16 21:50:56.196592 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.196576    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.199402 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.199381    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.208169 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.208139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-default-certificate\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.208293 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.208189    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ngtz\" (UniqueName: \"kubernetes.io/projected/216a31c0-40ba-4d6a-9c4f-fd734394fe44-kube-api-access-4ngtz\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.208293 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.208284    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.208398 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.208303    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-stats-auth\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.208398 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.208332    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.224164 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.224144    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:50:56.224531 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.224496    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 21:50:56.224638 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.224528    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c89j9\""
Jul 16 21:50:56.224638 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.224547    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 21:50:56.224748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.224656    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4b7tz\""
Jul 16 21:50:56.228899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.228883    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:50:56.230591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.230561    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:50:56.230705 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.230595    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 21:50:56.230705 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.230596    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 21:50:56.230817 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.230563    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 21:50:56.230975 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.230953    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 16 21:50:56.231068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.231023    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 16 21:50:56.231068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.231060    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 16 21:50:56.232268 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.232209    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5h8rq\""
Jul 16 21:50:56.233575 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.233533    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"]
Jul 16 21:50:56.239407 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.239379    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 16 21:50:56.243732 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.243703    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-5j647"]
Jul 16 21:50:56.259021 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.258994    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"]
Jul 16 21:50:56.309542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309461    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-default-certificate\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.309542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309502    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ngtz\" (UniqueName: \"kubernetes.io/projected/216a31c0-40ba-4d6a-9c4f-fd734394fe44-kube-api-access-4ngtz\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.309542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309530    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-serving-cert\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.309811 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309667    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvvb\" (UniqueName: \"kubernetes.io/projected/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-kube-api-access-4cvvb\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.309811 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309704    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.309811 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309743    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-646pj\" (UniqueName: \"kubernetes.io/projected/dab839eb-23d2-4353-a34d-fe726e14454b-kube-api-access-646pj\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.309811 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309788    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.309969 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309867    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.309969 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309895    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-config\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.309969 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-stats-auth\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.309969 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309933    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5p7\" (UniqueName: \"kubernetes.io/projected/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-kube-api-access-xt5p7\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.309969 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.309953    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:50:56.310200 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309969    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.310200 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.309994    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-trusted-ca\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.310200 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.310005    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:56.80998917 +0000 UTC m=+40.112161826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : secret "router-metrics-certs-default" not found
Jul 16 21:50:56.310200 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.310031    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.310200 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.310107    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:56.810090787 +0000 UTC m=+40.112263460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : configmap references non-existent config key: service-ca.crt
Jul 16 21:50:56.312683 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.312663    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-stats-auth\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.312784 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.312709    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-default-certificate\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.334810 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.334774    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ngtz\" (UniqueName: \"kubernetes.io/projected/216a31c0-40ba-4d6a-9c4f-fd734394fe44-kube-api-access-4ngtz\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.410627 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410586    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-config\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.410627 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410625    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5p7\" (UniqueName: \"kubernetes.io/projected/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-kube-api-access-xt5p7\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410663    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-trusted-ca\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410684    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410746    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-serving-cert\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410772    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvvb\" (UniqueName: \"kubernetes.io/projected/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-kube-api-access-4cvvb\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410797    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.410865 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410834    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-646pj\" (UniqueName: \"kubernetes.io/projected/dab839eb-23d2-4353-a34d-fe726e14454b-kube-api-access-646pj\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.411163 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.410877    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.411456 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.411428    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:50:56.411558 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.411501    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls podName:dab839eb-23d2-4353-a34d-fe726e14454b nodeName:}" failed. No retries permitted until 2026-07-16 21:50:56.91148291 +0000 UTC m=+40.213655561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fhvzg" (UID: "dab839eb-23d2-4353-a34d-fe726e14454b") : secret "samples-operator-tls" not found
Jul 16 21:50:56.411642 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.411564    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.411728 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.411697    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-trusted-ca\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.412087 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.412068    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-config\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.413599 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.413577    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.413699 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.413682    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-serving-cert\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.430126 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.430100    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-646pj\" (UniqueName: \"kubernetes.io/projected/dab839eb-23d2-4353-a34d-fe726e14454b-kube-api-access-646pj\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.430272 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.430204    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvvb\" (UniqueName: \"kubernetes.io/projected/e38ec207-88d3-4fd1-9e4f-ec9826d265b3-kube-api-access-4cvvb\") pod \"console-operator-565b7b569d-5j647\" (UID: \"e38ec207-88d3-4fd1-9e4f-ec9826d265b3\") " pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.432337 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.432318    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5p7\" (UniqueName: \"kubernetes.io/projected/1ae625e3-3f8b-40e5-aff6-4bb2a46115c9-kube-api-access-xt5p7\") pod \"kube-storage-version-migrator-operator-7658d9b749-6gl8q\" (UID: \"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.506493 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.506457    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:50:56.512084 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.512064    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"
Jul 16 21:50:56.648562 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.648531    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lxh7b" event={"ID":"59cc2fbf-efaf-4b09-870a-13472c7113c9","Type":"ContainerStarted","Data":"804be89969f973d3fc17bf7543f6fdc290e6e4f378009fba5388cd62156a1b05"}
Jul 16 21:50:56.652587 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.652557    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-5j647"]
Jul 16 21:50:56.655920 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:56.655898    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38ec207_88d3_4fd1_9e4f_ec9826d265b3.slice/crio-349d52c12c1433f0c6627dc3b3b17646c4e837ea18c70cd30ad77da6781831d4 WatchSource:0}: Error finding container 349d52c12c1433f0c6627dc3b3b17646c4e837ea18c70cd30ad77da6781831d4: Status 404 returned error can't find the container with id 349d52c12c1433f0c6627dc3b3b17646c4e837ea18c70cd30ad77da6781831d4
Jul 16 21:50:56.658133 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.658083    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q"]
Jul 16 21:50:56.660846 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:56.660826    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae625e3_3f8b_40e5_aff6_4bb2a46115c9.slice/crio-07ab3af9197bc1ed98f52cee0c64e77708a390f7b8b77eeda8f5cf3f2b02dfb9 WatchSource:0}: Error finding container 07ab3af9197bc1ed98f52cee0c64e77708a390f7b8b77eeda8f5cf3f2b02dfb9: Status 404 returned error can't find the container with id 07ab3af9197bc1ed98f52cee0c64e77708a390f7b8b77eeda8f5cf3f2b02dfb9
Jul 16 21:50:56.672025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.671981    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lxh7b" podStartSLOduration=1.073756512 podStartE2EDuration="3.671967167s" totalImagesPullingTime="2.598210655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:56.671735411 +0000 UTC m=+39.973908084" watchObservedRunningTime="2026-07-16 21:50:56.671967167 +0000 UTC m=+39.974139819"
Jul 16 21:50:56.813753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.813676    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.813753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.813738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:56.813955 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.813777    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:50:56.813955 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.813829    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:50:56.813955 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.813905    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:57.813885913 +0000 UTC m=+41.116058581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : secret "router-metrics-certs-default" not found
Jul 16 21:50:56.813955 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.813921    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:57.813913769 +0000 UTC m=+41.116086423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : configmap references non-existent config key: service-ca.crt
Jul 16 21:50:56.813955 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.813921    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:50:56.814128 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.813975    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls podName:61b46972-876c-430c-b9de-187f8f9e9c81 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:00.813957346 +0000 UTC m=+44.116129997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls") pod "dns-default-km54j" (UID: "61b46972-876c-430c-b9de-187f8f9e9c81") : secret "dns-default-metrics-tls" not found
Jul 16 21:50:56.914483 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:56.914441    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:56.914639 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.914549    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:50:56.914639 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:56.914616    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls podName:dab839eb-23d2-4353-a34d-fe726e14454b nodeName:}" failed. No retries permitted until 2026-07-16 21:50:57.91459969 +0000 UTC m=+41.216772340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fhvzg" (UID: "dab839eb-23d2-4353-a34d-fe726e14454b") : secret "samples-operator-tls" not found
Jul 16 21:50:57.173495 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.173416    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"]
Jul 16 21:50:57.204071 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.204039    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"]
Jul 16 21:50:57.204198 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.204137    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"
Jul 16 21:50:57.207305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.207284    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r6dpj\""
Jul 16 21:50:57.317412 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.317370    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/15b38079-2e8b-4b95-a15b-5db705392552-kube-api-access-q5mzp\") pod \"network-check-source-7bc5b6bf85-5zwt4\" (UID: \"15b38079-2e8b-4b95-a15b-5db705392552\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"
Jul 16 21:50:57.418851 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.418815    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/15b38079-2e8b-4b95-a15b-5db705392552-kube-api-access-q5mzp\") pod \"network-check-source-7bc5b6bf85-5zwt4\" (UID: \"15b38079-2e8b-4b95-a15b-5db705392552\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"
Jul 16 21:50:57.429433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.429401    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/15b38079-2e8b-4b95-a15b-5db705392552-kube-api-access-q5mzp\") pod \"network-check-source-7bc5b6bf85-5zwt4\" (UID: \"15b38079-2e8b-4b95-a15b-5db705392552\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"
Jul 16 21:50:57.512436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.512405    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"
Jul 16 21:50:57.652593 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.652559    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-5j647" event={"ID":"e38ec207-88d3-4fd1-9e4f-ec9826d265b3","Type":"ContainerStarted","Data":"349d52c12c1433f0c6627dc3b3b17646c4e837ea18c70cd30ad77da6781831d4"}
Jul 16 21:50:57.653781 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.653752    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q" event={"ID":"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9","Type":"ContainerStarted","Data":"07ab3af9197bc1ed98f52cee0c64e77708a390f7b8b77eeda8f5cf3f2b02dfb9"}
Jul 16 21:50:57.672373 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.672344    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4"]
Jul 16 21:50:57.675669 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:50:57.675633    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b38079_2e8b_4b95_a15b_5db705392552.slice/crio-3cf17638e876403c4cbc29f2879040b9a46fccffe64582f5e6a0559d0cada431 WatchSource:0}: Error finding container 3cf17638e876403c4cbc29f2879040b9a46fccffe64582f5e6a0559d0cada431: Status 404 returned error can't find the container with id 3cf17638e876403c4cbc29f2879040b9a46fccffe64582f5e6a0559d0cada431
Jul 16 21:50:57.822748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.822710    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:57.822914 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:57.822876    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:59.822855358 +0000 UTC m=+43.125028009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : configmap references non-existent config key: service-ca.crt
Jul 16 21:50:57.823084 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.823063    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:57.823201 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:57.823182    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:50:57.823298 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:57.823242    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:59.823224701 +0000 UTC m=+43.125397370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : secret "router-metrics-certs-default" not found
Jul 16 21:50:57.924282 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:57.924211    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:57.924455 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:57.924380    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:50:57.924455 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:57.924448    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls podName:dab839eb-23d2-4353-a34d-fe726e14454b nodeName:}" failed. No retries permitted until 2026-07-16 21:50:59.924430783 +0000 UTC m=+43.226603436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fhvzg" (UID: "dab839eb-23d2-4353-a34d-fe726e14454b") : secret "samples-operator-tls" not found
Jul 16 21:50:58.658180 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:58.658134    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4" event={"ID":"15b38079-2e8b-4b95-a15b-5db705392552","Type":"ContainerStarted","Data":"3cf17638e876403c4cbc29f2879040b9a46fccffe64582f5e6a0559d0cada431"}
Jul 16 21:50:59.839965 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:59.839926    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:59.840412 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:59.840078    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:50:59.840412 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:59.840104    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:03.840084089 +0000 UTC m=+47.142256746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : configmap references non-existent config key: service-ca.crt
Jul 16 21:50:59.840412 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:59.840166    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:50:59.840412 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:59.840198    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:03.840189325 +0000 UTC m=+47.142361979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : secret "router-metrics-certs-default" not found
Jul 16 21:50:59.941340 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:50:59.941313    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:50:59.941459 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:59.941431    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:50:59.941539 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:50:59.941521    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls podName:dab839eb-23d2-4353-a34d-fe726e14454b nodeName:}" failed. No retries permitted until 2026-07-16 21:51:03.941487218 +0000 UTC m=+47.243659872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fhvzg" (UID: "dab839eb-23d2-4353-a34d-fe726e14454b") : secret "samples-operator-tls" not found
Jul 16 21:51:00.664747 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.664694    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q" event={"ID":"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9","Type":"ContainerStarted","Data":"01a5ccf32e2af7ad3457cf541f671cba6dde51fc91ab1410db2b952270ebb1f4"}
Jul 16 21:51:00.666523 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.666491    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/0.log"
Jul 16 21:51:00.666656 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.666541    2551 generic.go:356] "Generic (PLEG): container finished" podID="e38ec207-88d3-4fd1-9e4f-ec9826d265b3" containerID="56fe723551d5acda936a90742782212c4bcf81d140e18e47bbee896cea3aad06" exitCode=255
Jul 16 21:51:00.666656 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.666576    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-5j647" event={"ID":"e38ec207-88d3-4fd1-9e4f-ec9826d265b3","Type":"ContainerDied","Data":"56fe723551d5acda936a90742782212c4bcf81d140e18e47bbee896cea3aad06"}
Jul 16 21:51:00.666830 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.666808    2551 scope.go:117] "RemoveContainer" containerID="56fe723551d5acda936a90742782212c4bcf81d140e18e47bbee896cea3aad06"
Jul 16 21:51:00.731542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.731479    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q" podStartSLOduration=1.460455811 podStartE2EDuration="4.731457478s" totalImagesPullingTime="3.271001667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:00.730525591 +0000 UTC m=+44.032698293" watchObservedRunningTime="2026-07-16 21:51:00.731457478 +0000 UTC m=+44.033630152"
Jul 16 21:51:00.848989 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:00.848941    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:00.849485 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:00.849146    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:51:00.849485 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:00.849211    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls podName:61b46972-876c-430c-b9de-187f8f9e9c81 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:08.849192704 +0000 UTC m=+52.151365373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls") pod "dns-default-km54j" (UID: "61b46972-876c-430c-b9de-187f8f9e9c81") : secret "dns-default-metrics-tls" not found
Jul 16 21:51:01.360288 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.360239    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jdmzl"]
Jul 16 21:51:01.376394 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.376364    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.380108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.380078    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jgk6d\""
Jul 16 21:51:01.380108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.380098    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 21:51:01.380108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.380105    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 21:51:01.380382 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.380145    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 21:51:01.380382 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.380098    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 21:51:01.389324 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.389298    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jdmzl"]
Jul 16 21:51:01.556103 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.556035    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8567367-27fd-45a5-b62b-016d87a108e9-crio-socket\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.556103 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.556070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.556103 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.556099    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662hb\" (UniqueName: \"kubernetes.io/projected/b8567367-27fd-45a5-b62b-016d87a108e9-kube-api-access-662hb\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.556306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.556200    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8567367-27fd-45a5-b62b-016d87a108e9-data-volume\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.556306 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.556293    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658501 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.658445    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-662hb\" (UniqueName: \"kubernetes.io/projected/b8567367-27fd-45a5-b62b-016d87a108e9-kube-api-access-662hb\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658668 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.658569    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8567367-27fd-45a5-b62b-016d87a108e9-data-volume\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658737 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.658667    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658794 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.658774    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8567367-27fd-45a5-b62b-016d87a108e9-crio-socket\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658840 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.658816    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.658955 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:01.658938    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:51:01.659075 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:01.659064    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:02.159042649 +0000 UTC m=+45.461215303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : configmap "kube-rbac-proxy" not found
Jul 16 21:51:01.661332 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:01.659407    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:01.661332 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:01.659494    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:02.159474523 +0000 UTC m=+45.461647200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:01.661332 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.659511    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8567367-27fd-45a5-b62b-016d87a108e9-crio-socket\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.661332 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.659796    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8567367-27fd-45a5-b62b-016d87a108e9-data-volume\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:01.670110 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.670090    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 21:51:01.670515 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.670500    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/0.log"
Jul 16 21:51:01.670594 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.670533    2551 generic.go:356] "Generic (PLEG): container finished" podID="e38ec207-88d3-4fd1-9e4f-ec9826d265b3" containerID="a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789" exitCode=255
Jul 16 21:51:01.670648 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.670621    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-5j647" event={"ID":"e38ec207-88d3-4fd1-9e4f-ec9826d265b3","Type":"ContainerDied","Data":"a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789"}
Jul 16 21:51:01.670722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.670709    2551 scope.go:117] "RemoveContainer" containerID="56fe723551d5acda936a90742782212c4bcf81d140e18e47bbee896cea3aad06"
Jul 16 21:51:01.671067 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.671048    2551 scope.go:117] "RemoveContainer" containerID="a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789"
Jul 16 21:51:01.671232 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:01.671211    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-5j647_openshift-console-operator(e38ec207-88d3-4fd1-9e4f-ec9826d265b3)\"" pod="openshift-console-operator/console-operator-565b7b569d-5j647" podUID="e38ec207-88d3-4fd1-9e4f-ec9826d265b3"
Jul 16 21:51:01.675353 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:01.675328    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-662hb\" (UniqueName: \"kubernetes.io/projected/b8567367-27fd-45a5-b62b-016d87a108e9-kube-api-access-662hb\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:02.164503 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.164415    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:02.164503 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.164498    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:02.165051 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:02.164568    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:51:02.165051 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:02.164569    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:02.165051 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:02.164620    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:03.164607311 +0000 UTC m=+46.466779962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : configmap "kube-rbac-proxy" not found
Jul 16 21:51:02.165051 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:02.164632    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:03.164626839 +0000 UTC m=+46.466799489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:02.674722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.674695    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 21:51:02.675109 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.675082    2551 scope.go:117] "RemoveContainer" containerID="a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789"
Jul 16 21:51:02.675310 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:02.675281    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-5j647_openshift-console-operator(e38ec207-88d3-4fd1-9e4f-ec9826d265b3)\"" pod="openshift-console-operator/console-operator-565b7b569d-5j647" podUID="e38ec207-88d3-4fd1-9e4f-ec9826d265b3"
Jul 16 21:51:02.676313 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.676287    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4" event={"ID":"15b38079-2e8b-4b95-a15b-5db705392552","Type":"ContainerStarted","Data":"299d8bb1fb16c12ff8052fe34fd16b7d3cb53ed83599c20e210123fc3b30aaeb"}
Jul 16 21:51:02.678672 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.678651    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="1ca428c1ff5e6e3bd7e0abeba941166168a500593093b0be689b168dab40ffdc" exitCode=0
Jul 16 21:51:02.678753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.678692    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"1ca428c1ff5e6e3bd7e0abeba941166168a500593093b0be689b168dab40ffdc"}
Jul 16 21:51:02.744561 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.744514    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zwt4" podStartSLOduration=1.6350357039999999 podStartE2EDuration="5.744497247s" totalImagesPullingTime="4.109461543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:02.74371082 +0000 UTC m=+46.045883518" watchObservedRunningTime="2026-07-16 21:51:02.744497247 +0000 UTC m=+46.046669921"
Jul 16 21:51:02.893992 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.893956    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"]
Jul 16 21:51:02.923542 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.923512    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"]
Jul 16 21:51:02.923726 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.923619    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"
Jul 16 21:51:02.926712 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.926638    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 21:51:02.926847 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.926648    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xkqxm\""
Jul 16 21:51:02.926847 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:02.926754    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 21:51:03.072697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.072660    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn2dw\" (UniqueName: \"kubernetes.io/projected/6ef76a2b-ea3d-43d1-b039-673fa844b967-kube-api-access-pn2dw\") pod \"migrator-7cc9c7ffd7-9ncq9\" (UID: \"6ef76a2b-ea3d-43d1-b039-673fa844b967\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"
Jul 16 21:51:03.174141 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.174106    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.174160    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn2dw\" (UniqueName: \"kubernetes.io/projected/6ef76a2b-ea3d-43d1-b039-673fa844b967-kube-api-access-pn2dw\") pod \"migrator-7cc9c7ffd7-9ncq9\" (UID: \"6ef76a2b-ea3d-43d1-b039-673fa844b967\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.174235    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.174333    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:05.174312491 +0000 UTC m=+48.476485154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : configmap "kube-rbac-proxy" not found
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.174364    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.174402    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:05.174390775 +0000 UTC m=+48.476563443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:03.174548 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.174278    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:03.187272 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.187189    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn2dw\" (UniqueName: \"kubernetes.io/projected/6ef76a2b-ea3d-43d1-b039-673fa844b967-kube-api-access-pn2dw\") pod \"migrator-7cc9c7ffd7-9ncq9\" (UID: \"6ef76a2b-ea3d-43d1-b039-673fa844b967\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"
Jul 16 21:51:03.233337 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.233301    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"
Jul 16 21:51:03.330402 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.330380    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n9tkf_d9552277-2e3a-421e-a0d1-4a3131769f8c/dns-node-resolver/0.log"
Jul 16 21:51:03.399401 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.398795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9"]
Jul 16 21:51:03.401774 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:03.401739    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef76a2b_ea3d_43d1_b039_673fa844b967.slice/crio-99ad27ad20fdb4a0cd20dd855124f55625d545af7b85cd7967852e307f5d3da2 WatchSource:0}: Error finding container 99ad27ad20fdb4a0cd20dd855124f55625d545af7b85cd7967852e307f5d3da2: Status 404 returned error can't find the container with id 99ad27ad20fdb4a0cd20dd855124f55625d545af7b85cd7967852e307f5d3da2
Jul 16 21:51:03.682503 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.682468    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9" event={"ID":"6ef76a2b-ea3d-43d1-b039-673fa844b967","Type":"ContainerStarted","Data":"99ad27ad20fdb4a0cd20dd855124f55625d545af7b85cd7967852e307f5d3da2"}
Jul 16 21:51:03.881675 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.881628    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:03.881857 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.881754    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:03.881857 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.881807    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:11.881786885 +0000 UTC m=+55.183959542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : configmap references non-existent config key: service-ca.crt
Jul 16 21:51:03.881985 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.881910    2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:51:03.881985 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.881980    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs podName:216a31c0-40ba-4d6a-9c4f-fd734394fe44 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:11.881962152 +0000 UTC m=+55.184134816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs") pod "router-default-748d5b8ddb-97qq5" (UID: "216a31c0-40ba-4d6a-9c4f-fd734394fe44") : secret "router-metrics-certs-default" not found
Jul 16 21:51:03.983083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:03.982996    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:51:03.983246 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.983185    2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:51:03.983329 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:03.983277    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls podName:dab839eb-23d2-4353-a34d-fe726e14454b nodeName:}" failed. No retries permitted until 2026-07-16 21:51:11.983242885 +0000 UTC m=+55.285415537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fhvzg" (UID: "dab839eb-23d2-4353-a34d-fe726e14454b") : secret "samples-operator-tls" not found
Jul 16 21:51:04.319957 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:04.319882    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nx9cb_b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91/node-ca/0.log"
Jul 16 21:51:04.925717 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:04.925690    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lxh7b_59cc2fbf-efaf-4b09-870a-13472c7113c9/serve-healthcheck-canary/0.log"
Jul 16 21:51:05.194511 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:05.194475    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:05.194641 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:05.194566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:05.194641 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:05.194609    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:05.194723 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:05.194675    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.194660563 +0000 UTC m=+52.496833230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:05.194723 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:05.194688    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:51:05.194817 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:05.194747    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.194729851 +0000 UTC m=+52.496902525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : configmap "kube-rbac-proxy" not found
Jul 16 21:51:05.689898 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:05.689858    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9" event={"ID":"6ef76a2b-ea3d-43d1-b039-673fa844b967","Type":"ContainerStarted","Data":"2345ff8eda3db8a6a352e82049ccc1c88a2e3a9e02eecae6d14f68f38ff5dd69"}
Jul 16 21:51:05.690330 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:05.689914    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9" event={"ID":"6ef76a2b-ea3d-43d1-b039-673fa844b967","Type":"ContainerStarted","Data":"e71c794d5aa90efd38cd4f8f1cb1549510a06a168ba1aef987f9f61668768571"}
Jul 16 21:51:05.720067 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:05.720006    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9ncq9" podStartSLOduration=2.129227499 podStartE2EDuration="3.71998615s" totalImagesPullingTime="1.590758651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:05.719423942 +0000 UTC m=+49.021596663" watchObservedRunningTime="2026-07-16 21:51:05.71998615 +0000 UTC m=+49.022158825"
Jul 16 21:51:06.507533 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:06.507500    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:51:06.507707 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:06.507567    2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:51:06.507962 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:06.507947    2551 scope.go:117] "RemoveContainer" containerID="a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789"
Jul 16 21:51:06.508141 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:06.508120    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-5j647_openshift-console-operator(e38ec207-88d3-4fd1-9e4f-ec9826d265b3)\"" pod="openshift-console-operator/console-operator-565b7b569d-5j647" podUID="e38ec207-88d3-4fd1-9e4f-ec9826d265b3"
Jul 16 21:51:07.698541 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:07.698509    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="135eb8222cab2cb265a91d5c85f40fed8a4a7db05e3f65e94a4091c47980cc6f" exitCode=0
Jul 16 21:51:07.698938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:07.698549    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"135eb8222cab2cb265a91d5c85f40fed8a4a7db05e3f65e94a4091c47980cc6f"}
Jul 16 21:51:08.703678 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:08.703643    2551 generic.go:356] "Generic (PLEG): container finished" podID="e027d183-8aec-4901-b982-4b2c88a803a8" containerID="eb8600dc7580f85b56829cfbcc9ee4a20661447c3c648ca2eb47ea551a83344f" exitCode=0
Jul 16 21:51:08.704048 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:08.703711    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerDied","Data":"eb8600dc7580f85b56829cfbcc9ee4a20661447c3c648ca2eb47ea551a83344f"}
Jul 16 21:51:08.927883 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:08.927848    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:08.930071 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:08.930049    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61b46972-876c-430c-b9de-187f8f9e9c81-metrics-tls\") pod \"dns-default-km54j\" (UID: \"61b46972-876c-430c-b9de-187f8f9e9c81\") " pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:08.959271 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:08.959192    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:09.104800 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.104769    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-km54j"]
Jul 16 21:51:09.107662 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:09.107631    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b46972_876c_430c_b9de_187f8f9e9c81.slice/crio-9864de8545ed956342e2049cd64efafd850554c36ecfe88cd3dbba85b6194ddf WatchSource:0}: Error finding container 9864de8545ed956342e2049cd64efafd850554c36ecfe88cd3dbba85b6194ddf: Status 404 returned error can't find the container with id 9864de8545ed956342e2049cd64efafd850554c36ecfe88cd3dbba85b6194ddf
Jul 16 21:51:09.230697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.230598    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:09.230697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.230671    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:09.230916 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:09.230775    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:51:09.230916 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:09.230777    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:09.230916 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:09.230827    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:17.230813247 +0000 UTC m=+60.532985897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : configmap "kube-rbac-proxy" not found
Jul 16 21:51:09.230916 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:09.230840    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls podName:b8567367-27fd-45a5-b62b-016d87a108e9 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:17.230834068 +0000 UTC m=+60.533006719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jdmzl" (UID: "b8567367-27fd-45a5-b62b-016d87a108e9") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:51:09.708212 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.708168    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km54j" event={"ID":"61b46972-876c-430c-b9de-187f8f9e9c81","Type":"ContainerStarted","Data":"9864de8545ed956342e2049cd64efafd850554c36ecfe88cd3dbba85b6194ddf"}
Jul 16 21:51:09.712180 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.712154    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" event={"ID":"e027d183-8aec-4901-b982-4b2c88a803a8","Type":"ContainerStarted","Data":"444a1544871461cb5e293350f2e8de9c666c57e9b1530a651e4c7f8ac3db0335"}
Jul 16 21:51:09.741853 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:09.741804    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dr7m8" podStartSLOduration=5.595433615 podStartE2EDuration="51.741789402s" totalImagesPullingTime="46.146355787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-16 21:51:09.739880531 +0000 UTC m=+53.042053204" watchObservedRunningTime="2026-07-16 21:51:09.741789402 +0000 UTC m=+53.043962074"
Jul 16 21:51:11.720409 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.720374    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km54j" event={"ID":"61b46972-876c-430c-b9de-187f8f9e9c81","Type":"ContainerStarted","Data":"b118c4f22f7a69c7dfe6acbae030e952d278919e70ef0e9e7832efdfd1b06aba"}
Jul 16 21:51:11.720842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.720456    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-km54j" event={"ID":"61b46972-876c-430c-b9de-187f8f9e9c81","Type":"ContainerStarted","Data":"4435a09eaf59dc23472637e05dba9ca04b91f95d1d224ce7af02b8ac702f52cc"}
Jul 16 21:51:11.720842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.720529    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:11.744859 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.744814    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-km54j" podStartSLOduration=16.883706356 podStartE2EDuration="18.74480103s" totalImagesPullingTime="1.861094674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:11.743364544 +0000 UTC m=+55.045537229" watchObservedRunningTime="2026-07-16 21:51:11.74480103 +0000 UTC m=+55.046973703"
Jul 16 21:51:11.954923 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.954873    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:11.955127 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.954975    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:11.955691 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.955671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/216a31c0-40ba-4d6a-9c4f-fd734394fe44-service-ca-bundle\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:11.957307 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.957285    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/216a31c0-40ba-4d6a-9c4f-fd734394fe44-metrics-certs\") pod \"router-default-748d5b8ddb-97qq5\" (UID: \"216a31c0-40ba-4d6a-9c4f-fd734394fe44\") " pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:11.963191 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:11.963173    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:12.055732 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.055701    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:51:12.058355 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.058324    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dab839eb-23d2-4353-a34d-fe726e14454b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fhvzg\" (UID: \"dab839eb-23d2-4353-a34d-fe726e14454b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:51:12.089549 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.089499    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-748d5b8ddb-97qq5"]
Jul 16 21:51:12.091908 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:12.091874    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216a31c0_40ba_4d6a_9c4f_fd734394fe44.slice/crio-dcf71b2f31b2fb02b705b4d00cc3f070c856919caec921ad8933cfb129b54914 WatchSource:0}: Error finding container dcf71b2f31b2fb02b705b4d00cc3f070c856919caec921ad8933cfb129b54914: Status 404 returned error can't find the container with id dcf71b2f31b2fb02b705b4d00cc3f070c856919caec921ad8933cfb129b54914
Jul 16 21:51:12.117138 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.117113    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"
Jul 16 21:51:12.271148 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.271122    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg"]
Jul 16 21:51:12.725673 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.725635    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg" event={"ID":"dab839eb-23d2-4353-a34d-fe726e14454b","Type":"ContainerStarted","Data":"8c60b6b7e4aa3d866463cb1825bd48333f9368ba3ee2449c9c350705623ed422"}
Jul 16 21:51:12.727081 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.727052    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-748d5b8ddb-97qq5" event={"ID":"216a31c0-40ba-4d6a-9c4f-fd734394fe44","Type":"ContainerStarted","Data":"78afc6b375e98ac52f126fcb123900273c9c95180dbea43895b124109e0ee2d8"}
Jul 16 21:51:12.727209 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.727104    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-748d5b8ddb-97qq5" event={"ID":"216a31c0-40ba-4d6a-9c4f-fd734394fe44","Type":"ContainerStarted","Data":"dcf71b2f31b2fb02b705b4d00cc3f070c856919caec921ad8933cfb129b54914"}
Jul 16 21:51:12.750513 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.750467    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-748d5b8ddb-97qq5" podStartSLOduration=16.750450461 podStartE2EDuration="16.750450461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:12.750348459 +0000 UTC m=+56.052521151" watchObservedRunningTime="2026-07-16 21:51:12.750450461 +0000 UTC m=+56.052623133"
Jul 16 21:51:12.964161 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.964094    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:12.966868 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:12.966840    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:13.731246 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:13.731215    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:13.732593 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:13.732568    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-748d5b8ddb-97qq5"
Jul 16 21:51:14.739678 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:14.739642    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg" event={"ID":"dab839eb-23d2-4353-a34d-fe726e14454b","Type":"ContainerStarted","Data":"c9242b1f48648c1b06e30d44873140f7b872a9cf6c9f85e485b4340642a44c62"}
Jul 16 21:51:14.740075 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:14.739732    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg" event={"ID":"dab839eb-23d2-4353-a34d-fe726e14454b","Type":"ContainerStarted","Data":"e7fae7876ea947a08fbda986893bd4307cf88bd72bb362b01e063a6ef0ea45d2"}
Jul 16 21:51:14.766244 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:14.766193    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fhvzg" podStartSLOduration=16.999508536 podStartE2EDuration="18.766178009s" totalImagesPullingTime="1.766669473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:14.765582155 +0000 UTC m=+58.067754827" watchObservedRunningTime="2026-07-16 21:51:14.766178009 +0000 UTC m=+58.068350695"
Jul 16 21:51:17.299571 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.299541    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:17.300016 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.299596    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:17.300068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.300027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8567367-27fd-45a5-b62b-016d87a108e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:17.301725 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.301703    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8567367-27fd-45a5-b62b-016d87a108e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdmzl\" (UID: \"b8567367-27fd-45a5-b62b-016d87a108e9\") " pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:17.589055 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.588975    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jgk6d\""
Jul 16 21:51:17.596163 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.596141    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jdmzl"
Jul 16 21:51:17.757868 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:17.757837    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jdmzl"]
Jul 16 21:51:17.760707 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:17.760678    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8567367_27fd_45a5_b62b_016d87a108e9.slice/crio-ff5bbee83dd3fbb7696558f3c73182bacaa8ebc2d780378d4cc9f1d1980808a0 WatchSource:0}: Error finding container ff5bbee83dd3fbb7696558f3c73182bacaa8ebc2d780378d4cc9f1d1980808a0: Status 404 returned error can't find the container with id ff5bbee83dd3fbb7696558f3c73182bacaa8ebc2d780378d4cc9f1d1980808a0
Jul 16 21:51:18.646741 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:18.646714    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t77lx"
Jul 16 21:51:18.754371 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:18.754325    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdmzl" event={"ID":"b8567367-27fd-45a5-b62b-016d87a108e9","Type":"ContainerStarted","Data":"ff5bbee83dd3fbb7696558f3c73182bacaa8ebc2d780378d4cc9f1d1980808a0"}
Jul 16 21:51:21.290349 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:21.290316    2551 scope.go:117] "RemoveContainer" containerID="a8ee989803ca46aaf0258dad7a970abd12e36d98ae65c889bd089b40ce791789"
Jul 16 21:51:21.729504 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:21.729464    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-km54j"
Jul 16 21:51:21.939175 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:21.939136    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:51:21.944990 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:21.944957    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:51:21.951852 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:21.951825    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5351fd34-a13a-4dc7-84a1-f68100b8c43c-metrics-certs\") pod \"network-metrics-daemon-2m5v5\" (UID: \"5351fd34-a13a-4dc7-84a1-f68100b8c43c\") " pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:51:22.040502 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.040419    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:51:22.040667 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.040503    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:51:22.043523 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.043490    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2sw8\" (UniqueName: \"kubernetes.io/projected/1e70a2b4-df90-4dfb-9a10-db3c438750dd-kube-api-access-g2sw8\") pod \"network-check-target-shstt\" (UID: \"1e70a2b4-df90-4dfb-9a10-db3c438750dd\") " pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:51:22.043918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.043896    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:51:22.054266 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.054223    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ab4a654-98ee-45e0-9284-62fe8e871a27-original-pull-secret\") pod \"global-pull-secret-syncer-l9vjn\" (UID: \"2ab4a654-98ee-45e0-9284-62fe8e871a27\") " pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:51:22.109618 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.109578    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9vjn"
Jul 16 21:51:22.117619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.117582    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qgzlt\""
Jul 16 21:51:22.122664 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.122633    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmhws\""
Jul 16 21:51:22.124921 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.124899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:51:22.130525 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:22.130504    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m5v5"
Jul 16 21:51:23.892869 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:23.892843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2m5v5"]
Jul 16 21:51:23.894027 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:23.893994    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5351fd34_a13a_4dc7_84a1_f68100b8c43c.slice/crio-f95f0caf725a04df5e5209d722a98acb3dbfdb231ab826333f7d850b86146b25 WatchSource:0}: Error finding container f95f0caf725a04df5e5209d722a98acb3dbfdb231ab826333f7d850b86146b25: Status 404 returned error can't find the container with id f95f0caf725a04df5e5209d722a98acb3dbfdb231ab826333f7d850b86146b25
Jul 16 21:51:23.921731 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:23.921698    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shstt"]
Jul 16 21:51:23.928181 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:23.928146    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e70a2b4_df90_4dfb_9a10_db3c438750dd.slice/crio-e9ffe6123e213b25ef77a32452934e71ebcf13a6ea4c15cdb1e9d3f05525666f WatchSource:0}: Error finding container e9ffe6123e213b25ef77a32452934e71ebcf13a6ea4c15cdb1e9d3f05525666f: Status 404 returned error can't find the container with id e9ffe6123e213b25ef77a32452934e71ebcf13a6ea4c15cdb1e9d3f05525666f
Jul 16 21:51:24.129859 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.129824    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9vjn"]
Jul 16 21:51:24.133652 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:24.133611    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab4a654_98ee_45e0_9284_62fe8e871a27.slice/crio-a52b1aa0d6c58f3a6b0b7321f6bdbb37c8da0715809fe91820ff2e57f954bfe5 WatchSource:0}: Error finding container a52b1aa0d6c58f3a6b0b7321f6bdbb37c8da0715809fe91820ff2e57f954bfe5: Status 404 returned error can't find the container with id a52b1aa0d6c58f3a6b0b7321f6bdbb37c8da0715809fe91820ff2e57f954bfe5
Jul 16 21:51:24.419651 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.418695    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-5xpz8"]
Jul 16 21:51:24.422085 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.422062    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:24.426384 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.425687    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 21:51:24.426384 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.426191    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-z9rr7\""
Jul 16 21:51:24.426384 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.426274    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 21:51:24.444174 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.443133    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5xpz8"]
Jul 16 21:51:24.560798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.560704    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkpmp\" (UniqueName: \"kubernetes.io/projected/00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6-kube-api-access-gkpmp\") pod \"downloads-5bc67dcd46-5xpz8\" (UID: \"00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6\") " pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:24.661767 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.661681    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpmp\" (UniqueName: \"kubernetes.io/projected/00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6-kube-api-access-gkpmp\") pod \"downloads-5bc67dcd46-5xpz8\" (UID: \"00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6\") " pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:24.675756 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.675700    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpmp\" (UniqueName: \"kubernetes.io/projected/00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6-kube-api-access-gkpmp\") pod \"downloads-5bc67dcd46-5xpz8\" (UID: \"00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6\") " pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:24.737336 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.737228    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:24.787108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.786367    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shstt" event={"ID":"1e70a2b4-df90-4dfb-9a10-db3c438750dd","Type":"ContainerStarted","Data":"542d4823577827fd0ecd1b864cce37c36abf3597a76cd3f08210df83f3245d65"}
Jul 16 21:51:24.787108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.786464    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shstt" event={"ID":"1e70a2b4-df90-4dfb-9a10-db3c438750dd","Type":"ContainerStarted","Data":"e9ffe6123e213b25ef77a32452934e71ebcf13a6ea4c15cdb1e9d3f05525666f"}
Jul 16 21:51:24.787108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.786596    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:51:24.788025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.787998    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m5v5" event={"ID":"5351fd34-a13a-4dc7-84a1-f68100b8c43c","Type":"ContainerStarted","Data":"f95f0caf725a04df5e5209d722a98acb3dbfdb231ab826333f7d850b86146b25"}
Jul 16 21:51:24.789964 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.789899    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9vjn" event={"ID":"2ab4a654-98ee-45e0-9284-62fe8e871a27","Type":"ContainerStarted","Data":"a52b1aa0d6c58f3a6b0b7321f6bdbb37c8da0715809fe91820ff2e57f954bfe5"}
Jul 16 21:51:24.795342 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.795224    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdmzl" event={"ID":"b8567367-27fd-45a5-b62b-016d87a108e9","Type":"ContainerStarted","Data":"af818cdc184d7d38b69f275126529e438a1104244abd40bbe3fc5d7885282d1f"}
Jul 16 21:51:24.801715 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.800029    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 21:51:24.801715 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.800095    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-5j647" event={"ID":"e38ec207-88d3-4fd1-9e4f-ec9826d265b3","Type":"ContainerStarted","Data":"e9f00b1660e45bfce55d3ab2b642abce106f074c6bb2b60d4366ccd07b347548"}
Jul 16 21:51:24.801715 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.801327    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:51:24.809744 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.809408    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-5j647"
Jul 16 21:51:24.836228 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.835628    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-shstt" podStartSLOduration=66.835605211 podStartE2EDuration="1m6.835605211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:24.807182732 +0000 UTC m=+68.109355432" watchObservedRunningTime="2026-07-16 21:51:24.835605211 +0000 UTC m=+68.137777887"
Jul 16 21:51:24.836228 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.835785    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-5j647" podStartSLOduration=25.554810587 podStartE2EDuration="28.835776128s" totalImagesPullingTime="3.280965541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:24.834122482 +0000 UTC m=+68.136295156" watchObservedRunningTime="2026-07-16 21:51:24.835776128 +0000 UTC m=+68.137948803"
Jul 16 21:51:24.930280 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:24.930164    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5xpz8"]
Jul 16 21:51:25.134676 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:25.134638    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ddb4c9_ebf5_4bb5_86cc_95c5360f0bb6.slice/crio-e4cfbf38f00acafbebb59389743d51383358abdc95ff1000c1e1aa616c4bd409 WatchSource:0}: Error finding container e4cfbf38f00acafbebb59389743d51383358abdc95ff1000c1e1aa616c4bd409: Status 404 returned error can't find the container with id e4cfbf38f00acafbebb59389743d51383358abdc95ff1000c1e1aa616c4bd409
Jul 16 21:51:25.809173 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:25.808336    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m5v5" event={"ID":"5351fd34-a13a-4dc7-84a1-f68100b8c43c","Type":"ContainerStarted","Data":"ece570330e7d3930d84550476f68925911e88dbe1e24a1496ccbd2316a3d9c8d"}
Jul 16 21:51:25.809173 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:25.808409    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m5v5" event={"ID":"5351fd34-a13a-4dc7-84a1-f68100b8c43c","Type":"ContainerStarted","Data":"2ccfa3928de889734df95e57826edd9c845c154e02c5f4517827b777e499e891"}
Jul 16 21:51:25.812066 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:25.812030    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5xpz8" event={"ID":"00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6","Type":"ContainerStarted","Data":"e4cfbf38f00acafbebb59389743d51383358abdc95ff1000c1e1aa616c4bd409"}
Jul 16 21:51:25.839643 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:25.839572    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2m5v5" podStartSLOduration=66.553030862 podStartE2EDuration="1m7.839549008s" totalImagesPullingTime="1.286518146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:25.838404623 +0000 UTC m=+69.140577297" watchObservedRunningTime="2026-07-16 21:51:25.839549008 +0000 UTC m=+69.141721682"
Jul 16 21:51:30.838312 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:30.838238    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9vjn" event={"ID":"2ab4a654-98ee-45e0-9284-62fe8e871a27","Type":"ContainerStarted","Data":"dc1c66bd534591c17d8de0c57e9df61102ee28dd5442b2b2baa8bb28afb9f454"}
Jul 16 21:51:30.840671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:30.840628    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdmzl" event={"ID":"b8567367-27fd-45a5-b62b-016d87a108e9","Type":"ContainerStarted","Data":"d2b7d29da6be5f202561345e83f00744c53a790104d815d976e43b935378b2bf"}
Jul 16 21:51:30.864456 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:30.864403    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l9vjn" podStartSLOduration=67.08214687 podStartE2EDuration="1m12.864381111s" totalImagesPullingTime="5.782234241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:30.862428059 +0000 UTC m=+74.164600732" watchObservedRunningTime="2026-07-16 21:51:30.864381111 +0000 UTC m=+74.166553785"
Jul 16 21:51:33.869076 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.869018    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:51:33.964367 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.964313    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:51:33.964556 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.964420    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:33.968502 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.968475    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 21:51:33.969912 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.969620    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 21:51:33.969912 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.969721    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 21:51:33.969912 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.969751    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 21:51:33.969912 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.969827    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r27qk\""
Jul 16 21:51:33.970141 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:33.970056    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 21:51:34.043170 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.043170 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043181    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.043420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043208    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.043420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043323    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.043420 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043415    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnhbj\" (UniqueName: \"kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.043540 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.043444    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144136    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnhbj\" (UniqueName: \"kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144184    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144446 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144225    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144446 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144276    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144446 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144306    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.144446 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144347    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.145025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144994    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.145135 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.144997    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.145135 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.145074    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.147024 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.146981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.147129 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.147064    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.153899 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.153873    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnhbj\" (UniqueName: \"kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj\") pod \"console-85cbd4dbfc-nfr2z\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") " pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.276828 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.276792    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:34.436666 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.436453    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:51:34.439357 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:34.439321    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba39197b_d2f0_4041_abf8_bad6333e7c0b.slice/crio-936e35b0fc8ff27adb73c3512516b0e21d6a80033886818d5249ed26853a357c WatchSource:0}: Error finding container 936e35b0fc8ff27adb73c3512516b0e21d6a80033886818d5249ed26853a357c: Status 404 returned error can't find the container with id 936e35b0fc8ff27adb73c3512516b0e21d6a80033886818d5249ed26853a357c
Jul 16 21:51:34.706363 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.706278    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"]
Jul 16 21:51:34.711286 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.711240    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.714232 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.714209    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 16 21:51:34.716368 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.716043    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 21:51:34.716368 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.716134    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 21:51:34.716368 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.716196    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xxpdm\""
Jul 16 21:51:34.716368 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.716285    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 21:51:34.716629 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.716544    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 16 21:51:34.723466 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.723445    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r5ks9"]
Jul 16 21:51:34.726645 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.726626    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.732183 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.732163    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 21:51:34.733233 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.733214    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 21:51:34.733330 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.733246    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 21:51:34.734582 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.734559    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dzgcx\""
Jul 16 21:51:34.798853 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.798815    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"]
Jul 16 21:51:34.802532 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.802509    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.806455 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.806429    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hmwgt\""
Jul 16 21:51:34.806602 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.806486    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 16 21:51:34.806897 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.806880    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 16 21:51:34.807240 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.807224    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 16 21:51:34.821798 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.821770    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"]
Jul 16 21:51:34.829611 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.829588    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"]
Jul 16 21:51:34.849152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.848957    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-sys\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849012    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-wtmp\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849050    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.849152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849074    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-textfile\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849112    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4tmn\" (UniqueName: \"kubernetes.io/projected/08310cbb-fe48-494c-96ad-ff0caf3fee73-kube-api-access-c4tmn\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849209    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849288    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-metrics-client-ca\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849324    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhpcn\" (UniqueName: \"kubernetes.io/projected/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-kube-api-access-dhpcn\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849351    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849378    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-root\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849445    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-tls\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.849555 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.849486    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.861133 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.861083    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdmzl" event={"ID":"b8567367-27fd-45a5-b62b-016d87a108e9","Type":"ContainerStarted","Data":"bc6d6c214c3736c928cdc3ddf0083f7ae1fcaac69fd19de03133b94bb4e42c4d"}
Jul 16 21:51:34.862725 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.862693    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cbd4dbfc-nfr2z" event={"ID":"ba39197b-d2f0-4041-abf8-bad6333e7c0b","Type":"ContainerStarted","Data":"936e35b0fc8ff27adb73c3512516b0e21d6a80033886818d5249ed26853a357c"}
Jul 16 21:51:34.916888 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.916297    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jdmzl" podStartSLOduration=17.774607546 podStartE2EDuration="33.916274195s" totalImagesPullingTime="16.141666649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:51:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:34.913075318 +0000 UTC m=+78.215248011" watchObservedRunningTime="2026-07-16 21:51:34.916274195 +0000 UTC m=+78.218446870"
Jul 16 21:51:34.950239 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950194    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.950437 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950266    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-textfile\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.950437 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950312    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.950437 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950343    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4tmn\" (UniqueName: \"kubernetes.io/projected/08310cbb-fe48-494c-96ad-ff0caf3fee73-kube-api-access-c4tmn\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.950437 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950376    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.950437 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950409    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nmtj\" (UniqueName: \"kubernetes.io/projected/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-api-access-5nmtj\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.950709 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950462    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.950709 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950492    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-metrics-client-ca\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.950810 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950787    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-textfile\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.950868 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.950785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c53ee81a-c8dc-4504-8448-bb268ee55ec5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.951104 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951062    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhpcn\" (UniqueName: \"kubernetes.io/projected/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-kube-api-access-dhpcn\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.951104 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951079    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951227 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951124    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951227 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951155    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.951227 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951185    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-root\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951364 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951242    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.951364 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951244    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-tls\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951396    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-root\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951400    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.951517 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951457    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.951517 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951491    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-sys\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951630 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951524    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-wtmp\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951630 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951548    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:34.951630 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951569    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08310cbb-fe48-494c-96ad-ff0caf3fee73-metrics-client-ca\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951785 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951632    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-sys\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951785 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:34.951696    2551 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 16 21:51:34.951785 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.951708    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-wtmp\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.951785 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:34.951767    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls podName:8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:35.451746405 +0000 UTC m=+78.753919073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-vbt6x" (UID: "8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3") : secret "openshift-state-metrics-tls" not found
Jul 16 21:51:34.954189 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.954154    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-tls\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.954342 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.954223    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08310cbb-fe48-494c-96ad-ff0caf3fee73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:34.954447 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.954426    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.965801 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.965731    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhpcn\" (UniqueName: \"kubernetes.io/projected/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-kube-api-access-dhpcn\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:34.967414 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:34.967388    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4tmn\" (UniqueName: \"kubernetes.io/projected/08310cbb-fe48-494c-96ad-ff0caf3fee73-kube-api-access-c4tmn\") pod \"node-exporter-r5ks9\" (UID: \"08310cbb-fe48-494c-96ad-ff0caf3fee73\") " pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:35.040529 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.040486    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5ks9"
Jul 16 21:51:35.052528 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052506    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.052610 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052577    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.052671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052622    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.052724 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052666    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nmtj\" (UniqueName: \"kubernetes.io/projected/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-api-access-5nmtj\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.052724 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052703    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.052829 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.052734    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c53ee81a-c8dc-4504-8448-bb268ee55ec5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.053856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.053108    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c53ee81a-c8dc-4504-8448-bb268ee55ec5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.054226 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.054176    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.054779 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.054750    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c53ee81a-c8dc-4504-8448-bb268ee55ec5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.056635 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.056555    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.056760 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.056735    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.070882 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.070851    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nmtj\" (UniqueName: \"kubernetes.io/projected/c53ee81a-c8dc-4504-8448-bb268ee55ec5-kube-api-access-5nmtj\") pod \"kube-state-metrics-56c5594674-gqs8z\" (UID: \"c53ee81a-c8dc-4504-8448-bb268ee55ec5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.113599 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.113567    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"
Jul 16 21:51:35.456423 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.456386    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:35.459769 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.459743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vbt6x\" (UID: \"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:35.623083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.623050    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"
Jul 16 21:51:35.820853 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.820759    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:51:35.825102 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.825060    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.832084 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.832029    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 21:51:35.833275 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.832792    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 21:51:35.833275 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.832810    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 21:51:35.833275 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.833170    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 21:51:35.833607 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.833592    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k68q7\""
Jul 16 21:51:35.833839 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.833788    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 21:51:35.834160 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.834002    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 21:51:35.834160 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.834042    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 21:51:35.834160 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.834055    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 21:51:35.834160 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.834080    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 21:51:35.855534 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.855088    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:51:35.869020 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.868980    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5ks9" event={"ID":"08310cbb-fe48-494c-96ad-ff0caf3fee73","Type":"ContainerStarted","Data":"2a5fcc7833f537bf4509a44b9edfef82514573550e1e793aaa5afe70ae4ea4ab"}
Jul 16 21:51:35.959393 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959356    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959393 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959397    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtn5\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959437    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959457    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959544    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959596    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959634    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959675    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959715    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959748    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959798    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959824    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:35.959926 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:35.959861    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.060898 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.060858    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.060909    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtn5\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.060941    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.060968    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.060997    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061028    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061091 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061056    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061096    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061125    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061154    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061187    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061215    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061239    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.061433 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.061296    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.062174 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:36.061900    2551 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 16 21:51:36.062174 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:51:36.061979    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls podName:b67ab509-c56d-43db-8c72-de6c65afaebe nodeName:}" failed. No retries permitted until 2026-07-16 21:51:36.561958629 +0000 UTC m=+79.864131286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe") : secret "alertmanager-main-tls" not found
Jul 16 21:51:36.062174 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.062055    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.062174 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.062140    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.065411 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.065011    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.065411 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.065113    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.065411 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.065372    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.066196 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.066168    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.066792 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.066753    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.067197 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.067170    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.067670 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.067607    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.068270 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.068150    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.083613 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.083545    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtn5\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.566512 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.566476    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.569218 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.569197    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.737551 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.737504    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:51:36.752987 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.752954    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c86559bf-zqm2v"]
Jul 16 21:51:36.756748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.756724    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.761150 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.761097    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 16 21:51:36.761150 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.761123    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 16 21:51:36.761748 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.761617    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 16 21:51:36.761938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.761918    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 16 21:51:36.761938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.761930    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-69mq6\""
Jul 16 21:51:36.762055 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.762040    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d5ijuqge713in\""
Jul 16 21:51:36.762551 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.762532    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 16 21:51:36.770706 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.770672    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c86559bf-zqm2v"]
Jul 16 21:51:36.868543 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868443    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868543 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868494    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868773 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868599    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-grpc-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868773 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868639    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868773 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868710    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868907 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868771    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eb2219d-128e-4d69-aadc-5f837c09a6cc-metrics-client-ca\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868907 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868799    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdz86\" (UniqueName: \"kubernetes.io/projected/8eb2219d-128e-4d69-aadc-5f837c09a6cc-kube-api-access-vdz86\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.868907 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.868867    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969314 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969278    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-grpc-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969366    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969434    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969479    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eb2219d-128e-4d69-aadc-5f837c09a6cc-metrics-client-ca\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969508    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdz86\" (UniqueName: \"kubernetes.io/projected/8eb2219d-128e-4d69-aadc-5f837c09a6cc-kube-api-access-vdz86\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969546    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.969808 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.969585    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.970386 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.970350    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eb2219d-128e-4d69-aadc-5f837c09a6cc-metrics-client-ca\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.972697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.972656    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.972697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.972665    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-grpc-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.972821 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.972705    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-tls\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.972821 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.972775    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.973107 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.973090    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.973216 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.973195    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8eb2219d-128e-4d69-aadc-5f837c09a6cc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:36.979454 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:36.979430    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdz86\" (UniqueName: \"kubernetes.io/projected/8eb2219d-128e-4d69-aadc-5f837c09a6cc-kube-api-access-vdz86\") pod \"thanos-querier-6c86559bf-zqm2v\" (UID: \"8eb2219d-128e-4d69-aadc-5f837c09a6cc\") " pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:37.068635 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:37.068596    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:41.277971 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.277851    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:51:41.312022 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.311987    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.318361 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.318335    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:51:41.323083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323063    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 21:51:41.323231 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323208    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 21:51:41.323358 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323331    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 21:51:41.323422 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323080    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 21:51:41.323422 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323145    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 21:51:41.323422 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323191    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 21:51:41.323697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323654    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 21:51:41.323697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323667    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 21:51:41.323697 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.323681    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xjrcq\""
Jul 16 21:51:41.324108 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324089    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 21:51:41.324493 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324472    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 21:51:41.324587 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324500    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 21:51:41.324587 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324545    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 21:51:41.324706 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324668    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 21:51:41.324752 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.324708    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e086k6jp8d013\""
Jul 16 21:51:41.326872 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.326825    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 21:51:41.329012 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.328994    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 21:51:41.414885 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.414848    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.414897    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.414929    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.414958    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.414977    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415000    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415068 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415023    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415170    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415207    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx7t8\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415338    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415381    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415461 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415446    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415489    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415523    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415550    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415605    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415637    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415673    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.415753 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.415692    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516735 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516693    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516748    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516776    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx7t8\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516815    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516844    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516871    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.516918 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516902    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516927    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.516963    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517002    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517044    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517088    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517115    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517142    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517237 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517199    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517269    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517309    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517335    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517358    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517671 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517383    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.517917 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517775    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.518004 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.517977    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.518679 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.518334    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.520266 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.520148    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.520541    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.520598    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.520761    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521152 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.520859    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521399 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.521181    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521469 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.521447    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521549 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.521485    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521796 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.521740    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.521995 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.521952    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.523355 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.523296    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.523757 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.523712    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.524096 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.524074    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.524455 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.524433    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.524714 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.524693    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.525335 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.525270    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.528668 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.528618    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx7t8\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8\") pod \"prometheus-k8s-0\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:41.626170 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:41.626130    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:51:44.176839 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.176765    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x"]
Jul 16 21:51:44.272110 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.271997    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gqs8z"]
Jul 16 21:51:44.348504 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:44.348472    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd87e0f_4c24_4cbc_921e_bd7c5164bbb3.slice/crio-411139c4f197d58dad8587c031cfd02f11a08ea64082a9e6407739d33a505fdf WatchSource:0}: Error finding container 411139c4f197d58dad8587c031cfd02f11a08ea64082a9e6407739d33a505fdf: Status 404 returned error can't find the container with id 411139c4f197d58dad8587c031cfd02f11a08ea64082a9e6407739d33a505fdf
Jul 16 21:51:44.349712 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:44.349687    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53ee81a_c8dc_4504_8448_bb268ee55ec5.slice/crio-f6a781040e1e77915dba6f5d5de4667e40afdad92b06f8eeb3873d3c7403f7c3 WatchSource:0}: Error finding container f6a781040e1e77915dba6f5d5de4667e40afdad92b06f8eeb3873d3c7403f7c3: Status 404 returned error can't find the container with id f6a781040e1e77915dba6f5d5de4667e40afdad92b06f8eeb3873d3c7403f7c3
Jul 16 21:51:44.419431 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.419402    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c86559bf-zqm2v"]
Jul 16 21:51:44.421787 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:44.421754    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb2219d_128e_4d69_aadc_5f837c09a6cc.slice/crio-814b65a2802f8c4ea83ffcb8f19bfd24309c7fc1aa0feac89d755299eea3203a WatchSource:0}: Error finding container 814b65a2802f8c4ea83ffcb8f19bfd24309c7fc1aa0feac89d755299eea3203a: Status 404 returned error can't find the container with id 814b65a2802f8c4ea83ffcb8f19bfd24309c7fc1aa0feac89d755299eea3203a
Jul 16 21:51:44.431424 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.431337    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:51:44.435578 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:44.435545    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67ab509_c56d_43db_8c72_de6c65afaebe.slice/crio-61ef88d5143c7202f8279240bf0e131d92f65d83ebdb49cd239c391b765543be WatchSource:0}: Error finding container 61ef88d5143c7202f8279240bf0e131d92f65d83ebdb49cd239c391b765543be: Status 404 returned error can't find the container with id 61ef88d5143c7202f8279240bf0e131d92f65d83ebdb49cd239c391b765543be
Jul 16 21:51:44.436667 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.436645    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:51:44.437653 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:51:44.437574    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d9d265_dfbf_426e_910d_fdc95a951560.slice/crio-27e33f7df9c39f8acf23c86f671d00ab9f90b42d68a03ca775274478c777f1f2 WatchSource:0}: Error finding container 27e33f7df9c39f8acf23c86f671d00ab9f90b42d68a03ca775274478c777f1f2: Status 404 returned error can't find the container with id 27e33f7df9c39f8acf23c86f671d00ab9f90b42d68a03ca775274478c777f1f2
Jul 16 21:51:44.918043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.916319    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5xpz8" event={"ID":"00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6","Type":"ContainerStarted","Data":"ccbc3534c69a2608f1fd4aac2d1dc443a7a9211457f2f6187e37f6d2aa3e0be2"}
Jul 16 21:51:44.918043 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.917949    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:44.923023 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.922990    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x" event={"ID":"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3","Type":"ContainerStarted","Data":"4ba10d322bc22af4cce4d67eb140ae9120c2246357c91423528dcafff667d0b8"}
Jul 16 21:51:44.923222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.923049    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x" event={"ID":"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3","Type":"ContainerStarted","Data":"4dd64ec305801f6aec3ed948dab0e43d3f32227dae1add0c2afef0c7700aea35"}
Jul 16 21:51:44.923222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.923078    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x" event={"ID":"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3","Type":"ContainerStarted","Data":"411139c4f197d58dad8587c031cfd02f11a08ea64082a9e6407739d33a505fdf"}
Jul 16 21:51:44.924960 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.924932    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"27e33f7df9c39f8acf23c86f671d00ab9f90b42d68a03ca775274478c777f1f2"}
Jul 16 21:51:44.928588 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.928534    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cbd4dbfc-nfr2z" event={"ID":"ba39197b-d2f0-4041-abf8-bad6333e7c0b","Type":"ContainerStarted","Data":"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"}
Jul 16 21:51:44.930821 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.930796    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"814b65a2802f8c4ea83ffcb8f19bfd24309c7fc1aa0feac89d755299eea3203a"}
Jul 16 21:51:44.936660 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.936499    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-5xpz8"
Jul 16 21:51:44.939317 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.938915    2551 generic.go:356] "Generic (PLEG): container finished" podID="08310cbb-fe48-494c-96ad-ff0caf3fee73" containerID="f49403e731b89fa6cefbc73bc94332c05647ca13d95ac3dfc520e7ef01488b8e" exitCode=0
Jul 16 21:51:44.939317 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.938988    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5ks9" event={"ID":"08310cbb-fe48-494c-96ad-ff0caf3fee73","Type":"ContainerDied","Data":"f49403e731b89fa6cefbc73bc94332c05647ca13d95ac3dfc520e7ef01488b8e"}
Jul 16 21:51:44.941113 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.941044    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z" event={"ID":"c53ee81a-c8dc-4504-8448-bb268ee55ec5","Type":"ContainerStarted","Data":"f6a781040e1e77915dba6f5d5de4667e40afdad92b06f8eeb3873d3c7403f7c3"}
Jul 16 21:51:44.943429 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.943350    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"61ef88d5143c7202f8279240bf0e131d92f65d83ebdb49cd239c391b765543be"}
Jul 16 21:51:44.959971 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:44.959637    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-5xpz8" podStartSLOduration=2.094581324 podStartE2EDuration="20.9596172s" totalImagesPullingTime="18.865035876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:44.958481232 +0000 UTC m=+88.260653906" watchObservedRunningTime="2026-07-16 21:51:44.9596172 +0000 UTC m=+88.261789875"
Jul 16 21:51:45.090272 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:45.090197    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85cbd4dbfc-nfr2z" podStartSLOduration=2.553807595 podStartE2EDuration="12.090170978s" totalImagesPullingTime="9.536363383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:45.074635216 +0000 UTC m=+88.376807918" watchObservedRunningTime="2026-07-16 21:51:45.090170978 +0000 UTC m=+88.392343652"
Jul 16 21:51:45.955896 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:45.955858    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5ks9" event={"ID":"08310cbb-fe48-494c-96ad-ff0caf3fee73","Type":"ContainerStarted","Data":"aa5f309022045a57a4cd98a6b8a822c7749f350037137155561084a5645e890e"}
Jul 16 21:51:45.956361 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:45.955934    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5ks9" event={"ID":"08310cbb-fe48-494c-96ad-ff0caf3fee73","Type":"ContainerStarted","Data":"5812a8434acd15aa550c238eb47482e04569e2eea0d55539b7bd1cf9346a7ab7"}
Jul 16 21:51:45.990087 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:45.988968    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r5ks9" podStartSLOduration=2.581788564 podStartE2EDuration="11.988943546s" totalImagesPullingTime="9.331106879s" totalInitContainerRuntime="76.048103ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:45.987976613 +0000 UTC m=+89.290149308" watchObservedRunningTime="2026-07-16 21:51:45.988943546 +0000 UTC m=+89.291116222"
Jul 16 21:51:48.972148 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.972119    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879" exitCode=0
Jul 16 21:51:48.972791 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.972670    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879"}
Jul 16 21:51:48.976623 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.976553    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"b9175bd7c1d0ea31f21fe0f6d2cf2c7157d4a21d9de10cf09ccc52669a5d3407"}
Jul 16 21:51:48.976745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.976661    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"0b12c3399c455dd48f01a0b76c6418acbe4a691b953de2b58afaeac3094de21b"}
Jul 16 21:51:48.976745 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.976720    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"2293b6a957d73dda98272527270c19b3eac40fa9a7cb3544f5c096884ce80c5c"}
Jul 16 21:51:48.980660 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.979663    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z" event={"ID":"c53ee81a-c8dc-4504-8448-bb268ee55ec5","Type":"ContainerStarted","Data":"cd38d10e7782629f095827d3961a825083bebb57a3865d83220dd152e010ee99"}
Jul 16 21:51:48.980660 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.979723    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z" event={"ID":"c53ee81a-c8dc-4504-8448-bb268ee55ec5","Type":"ContainerStarted","Data":"e101daab09cd2e3597bbea087568abde6fd15e5593fb362ac1fa5a43e810f507"}
Jul 16 21:51:48.980660 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.979756    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z" event={"ID":"c53ee81a-c8dc-4504-8448-bb268ee55ec5","Type":"ContainerStarted","Data":"a2304689cf352be21d927e902a760ce6e7751bb25b787b70359e20f6eab68fd7"}
Jul 16 21:51:48.986028 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.985122    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b" exitCode=0
Jul 16 21:51:48.986028 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.985216    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"}
Jul 16 21:51:48.993622 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:48.992999    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x" event={"ID":"8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3","Type":"ContainerStarted","Data":"c0122ef81ea6c96c158efd804f28e3364294ae0602bb9ea2693420ca04088998"}
Jul 16 21:51:49.126308 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:49.126224    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vbt6x" podStartSLOduration=11.181974137 podStartE2EDuration="15.126200688s" totalImagesPullingTime="3.944226551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:49.093615103 +0000 UTC m=+92.395787779" watchObservedRunningTime="2026-07-16 21:51:49.126200688 +0000 UTC m=+92.428373362"
Jul 16 21:51:49.126590 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:49.126553    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-gqs8z" podStartSLOduration=11.036049194 podStartE2EDuration="15.126543349s" totalImagesPullingTime="4.090494155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:49.124175572 +0000 UTC m=+92.426348273" watchObservedRunningTime="2026-07-16 21:51:49.126543349 +0000 UTC m=+92.428716025"
Jul 16 21:51:52.021890 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:52.021845    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"b804605163be897f137bdb534126faa88deb7e64507eadce10de539ce6da5e02"}
Jul 16 21:51:53.030664 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:53.030625    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"a47a1effa9807366aa8a2e758b65b8413a0bf6bd750a28858fe23d3bfbb0811b"}
Jul 16 21:51:53.031075 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:53.030713    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" event={"ID":"8eb2219d-128e-4d69-aadc-5f837c09a6cc","Type":"ContainerStarted","Data":"42a5f9feada55f1979645995c2ce7c0661fb9429381b971e501f4eb9210da068"}
Jul 16 21:51:53.033049 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:53.033011    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"}
Jul 16 21:51:53.560117 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:53.560093    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:51:54.043579 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043488    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"}
Jul 16 21:51:54.044045 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043579    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"}
Jul 16 21:51:54.044045 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043623    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"}
Jul 16 21:51:54.044045 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043660    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"}
Jul 16 21:51:54.044045 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043698    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"}
Jul 16 21:51:54.044045 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.043742    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerStarted","Data":"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"}
Jul 16 21:51:54.046967 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.046933    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"}
Jul 16 21:51:54.047109 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.047011    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"}
Jul 16 21:51:54.047109 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.047056    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"}
Jul 16 21:51:54.047224 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.047094    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"}
Jul 16 21:51:54.047224 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.047163    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerStarted","Data":"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"}
Jul 16 21:51:54.081856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.081792    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.766761494 podStartE2EDuration="13.081771424s" totalImagesPullingTime="8.282325303s" totalInitContainerRuntime="32.684627ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:54.079864275 +0000 UTC m=+97.382036972" watchObservedRunningTime="2026-07-16 21:51:54.081771424 +0000 UTC m=+97.383944098"
Jul 16 21:51:54.143305 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.143223    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.355118307 podStartE2EDuration="19.143174566s" totalImagesPullingTime="7.764312204s" totalInitContainerRuntime="23.744055ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:54.118492585 +0000 UTC m=+97.420665258" watchObservedRunningTime="2026-07-16 21:51:54.143174566 +0000 UTC m=+97.445347240"
Jul 16 21:51:54.144075 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.144041    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v" podStartSLOduration=12.043671967 podStartE2EDuration="18.144030783s" totalImagesPullingTime="6.100358816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:54.141448538 +0000 UTC m=+97.443621230" watchObservedRunningTime="2026-07-16 21:51:54.144030783 +0000 UTC m=+97.446203456"
Jul 16 21:51:54.277363 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:54.277331    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:51:55.051810 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:55.051782    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:55.057047 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:55.057023    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c86559bf-zqm2v"
Jul 16 21:51:55.815682 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:55.815655    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-shstt"
Jul 16 21:51:56.626527 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:51:56.626500    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:52:18.599432 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.599391    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85cbd4dbfc-nfr2z" podUID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" containerName="console" containerID="cri-o://21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e" gracePeriod=15
Jul 16 21:52:18.873485 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.873394    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:52:18.966157 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966130    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966370 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966169    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966370 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966186    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966370 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966234    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnhbj\" (UniqueName: \"kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966370 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966315    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966370 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966352    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config\") pod \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\" (UID: \"ba39197b-d2f0-4041-abf8-bad6333e7c0b\") "
Jul 16 21:52:18.966666 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966629    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:18.966666 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966653    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca" (OuterVolumeSpecName: "service-ca") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:18.966830 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.966757    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config" (OuterVolumeSpecName: "console-config") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:18.968535 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.968509    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:18.968626 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.968548    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj" (OuterVolumeSpecName: "kube-api-access-qnhbj") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "kube-api-access-qnhbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:52:18.968626 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:18.968582    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ba39197b-d2f0-4041-abf8-bad6333e7c0b" (UID: "ba39197b-d2f0-4041-abf8-bad6333e7c0b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:19.067908 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067875    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-service-ca\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.067908 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067901    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.067908 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067910    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba39197b-d2f0-4041-abf8-bad6333e7c0b-oauth-serving-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.067908 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067919    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-oauth-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.068159 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067928    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba39197b-d2f0-4041-abf8-bad6333e7c0b-console-serving-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.068159 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.067938    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnhbj\" (UniqueName: \"kubernetes.io/projected/ba39197b-d2f0-4041-abf8-bad6333e7c0b-kube-api-access-qnhbj\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:19.149507 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.149418    2551 generic.go:356] "Generic (PLEG): container finished" podID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" containerID="21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e" exitCode=0
Jul 16 21:52:19.149507 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.149460    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cbd4dbfc-nfr2z"
Jul 16 21:52:19.149722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.149504    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cbd4dbfc-nfr2z" event={"ID":"ba39197b-d2f0-4041-abf8-bad6333e7c0b","Type":"ContainerDied","Data":"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"}
Jul 16 21:52:19.149722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.149552    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cbd4dbfc-nfr2z" event={"ID":"ba39197b-d2f0-4041-abf8-bad6333e7c0b","Type":"ContainerDied","Data":"936e35b0fc8ff27adb73c3512516b0e21d6a80033886818d5249ed26853a357c"}
Jul 16 21:52:19.149722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.149573    2551 scope.go:117] "RemoveContainer" containerID="21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"
Jul 16 21:52:19.158318 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.158298    2551 scope.go:117] "RemoveContainer" containerID="21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"
Jul 16 21:52:19.158587 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:19.158568    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e\": container with ID starting with 21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e not found: ID does not exist" containerID="21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"
Jul 16 21:52:19.158657 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.158595    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e"} err="failed to get container status \"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e\": rpc error: code = NotFound desc = could not find container \"21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e\": container with ID starting with 21c953848217267703ecbd92215153f1a5a889961431f6bdff5e4b368888db7e not found: ID does not exist"
Jul 16 21:52:19.170728 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.170703    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:52:19.174047 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.174024    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85cbd4dbfc-nfr2z"]
Jul 16 21:52:19.294673 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:19.294641    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" path="/var/lib/kubelet/pods/ba39197b-d2f0-4041-abf8-bad6333e7c0b/volumes"
Jul 16 21:52:31.199067 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:31.199031    2551 generic.go:356] "Generic (PLEG): container finished" podID="1ae625e3-3f8b-40e5-aff6-4bb2a46115c9" containerID="01a5ccf32e2af7ad3457cf541f671cba6dde51fc91ab1410db2b952270ebb1f4" exitCode=0
Jul 16 21:52:31.199505 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:31.199105    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q" event={"ID":"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9","Type":"ContainerDied","Data":"01a5ccf32e2af7ad3457cf541f671cba6dde51fc91ab1410db2b952270ebb1f4"}
Jul 16 21:52:31.199505 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:31.199486    2551 scope.go:117] "RemoveContainer" containerID="01a5ccf32e2af7ad3457cf541f671cba6dde51fc91ab1410db2b952270ebb1f4"
Jul 16 21:52:32.204859 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:32.204823    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6gl8q" event={"ID":"1ae625e3-3f8b-40e5-aff6-4bb2a46115c9","Type":"ContainerStarted","Data":"2ba4991fbb8e47337da3f102e7326c630c048d04cac1047c96227df1a13ab033"}
Jul 16 21:52:41.627333 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:41.627292    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:52:41.643902 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:41.643875    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:52:42.269413 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:42.269379    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:52:55.243754 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.243718    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244104    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="alertmanager" containerID="cri-o://5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512" gracePeriod=120
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244158    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-web" containerID="cri-o://f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc" gracePeriod=120
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244158    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-metric" containerID="cri-o://7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5" gracePeriod=120
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244207    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="prom-label-proxy" containerID="cri-o://4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554" gracePeriod=120
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244242    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy" containerID="cri-o://22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e" gracePeriod=120
Jul 16 21:52:55.244299 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:55.244269    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="config-reloader" containerID="cri-o://cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f" gracePeriod=120
Jul 16 21:52:56.317874 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317842    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554" exitCode=0
Jul 16 21:52:56.317874 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317872    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e" exitCode=0
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317882    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f" exitCode=0
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317893    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512" exitCode=0
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317911    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"}
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.317992    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"}
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.318027    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"}
Jul 16 21:52:56.318222 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.318053    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"}
Jul 16 21:52:56.495297 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.495266    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:56.604710 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604630    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604710 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604691    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604714    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604733    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604762    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604786    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.604938 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604892    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.604970    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605024    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjtn5\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605054    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605107    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605142    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:52:56.605184 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605154    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605524 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605212    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:56.605524 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605233    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web\") pod \"b67ab509-c56d-43db-8c72-de6c65afaebe\" (UID: \"b67ab509-c56d-43db-8c72-de6c65afaebe\") "
Jul 16 21:52:56.605638 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605545    2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.605638 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605567    2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-alertmanager-main-db\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.606094 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.605795    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:56.607804 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.607761    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume" (OuterVolumeSpecName: "config-volume") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.607921 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.607883    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5" (OuterVolumeSpecName: "kube-api-access-tjtn5") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "kube-api-access-tjtn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:52:56.608269 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.608197    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.608363 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.608267    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.608452 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.608428    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:52:56.608816 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.608785    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.608816 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.608801    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.609840 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.609818    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out" (OuterVolumeSpecName: "config-out") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:52:56.612906 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.612858    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.619379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.619358    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config" (OuterVolumeSpecName: "web-config") pod "b67ab509-c56d-43db-8c72-de6c65afaebe" (UID: "b67ab509-c56d-43db-8c72-de6c65afaebe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:52:56.705947 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705908    2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.705947 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705941    2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-tls-assets\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.705947 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705951    2551 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-config-volume\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.705947 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705959    2551 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-cluster-tls-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705969    2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705978    2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705989    2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b67ab509-c56d-43db-8c72-de6c65afaebe-config-out\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.705997    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjtn5\" (UniqueName: \"kubernetes.io/projected/b67ab509-c56d-43db-8c72-de6c65afaebe-kube-api-access-tjtn5\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.706004    2551 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ab509-c56d-43db-8c72-de6c65afaebe-metrics-client-ca\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.706013    2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-web-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:56.706214 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:56.706021    2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b67ab509-c56d-43db-8c72-de6c65afaebe-secret-alertmanager-main-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:52:57.327906 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.327869    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5" exitCode=0
Jul 16 21:52:57.327906 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.327899    2551 generic.go:356] "Generic (PLEG): container finished" podID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerID="f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc" exitCode=0
Jul 16 21:52:57.328379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.327951    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.328379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.327950    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"}
Jul 16 21:52:57.328379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.328041    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"}
Jul 16 21:52:57.328379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.328089    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b67ab509-c56d-43db-8c72-de6c65afaebe","Type":"ContainerDied","Data":"61ef88d5143c7202f8279240bf0e131d92f65d83ebdb49cd239c391b765543be"}
Jul 16 21:52:57.328379 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.328120    2551 scope.go:117] "RemoveContainer" containerID="4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"
Jul 16 21:52:57.336611 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.336592    2551 scope.go:117] "RemoveContainer" containerID="7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"
Jul 16 21:52:57.344107 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.344088    2551 scope.go:117] "RemoveContainer" containerID="22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"
Jul 16 21:52:57.353479 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.353455    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:57.353795 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.353779    2551 scope.go:117] "RemoveContainer" containerID="f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"
Jul 16 21:52:57.357757 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.357731    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:57.363295 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.363267    2551 scope.go:117] "RemoveContainer" containerID="cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"
Jul 16 21:52:57.370403 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.370386    2551 scope.go:117] "RemoveContainer" containerID="5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"
Jul 16 21:52:57.377802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.377780    2551 scope.go:117] "RemoveContainer" containerID="6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"
Jul 16 21:52:57.384953 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.384930    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:57.385240 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385229    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="prom-label-proxy"
Jul 16 21:52:57.385298 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385242    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="prom-label-proxy"
Jul 16 21:52:57.385298 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385273    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="init-config-reloader"
Jul 16 21:52:57.385298 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385284    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="init-config-reloader"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385299    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-metric"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385305    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-metric"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385316    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385326    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385338    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="config-reloader"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385346    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="config-reloader"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385359    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" containerName="console"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385367    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" containerName="console"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385384    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="alertmanager"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385392    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="alertmanager"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385400    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-web"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385405    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-web"
Jul 16 21:52:57.385451 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385409    2551 scope.go:117] "RemoveContainer" containerID="4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385459    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-web"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385470    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="alertmanager"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385478    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba39197b-d2f0-4041-abf8-bad6333e7c0b" containerName="console"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385485    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385493    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="config-reloader"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385498    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="kube-rbac-proxy-metric"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385504    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" containerName="prom-label-proxy"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.385676    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554\": container with ID starting with 4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554 not found: ID does not exist" containerID="4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385699    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"} err="failed to get container status \"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554\": rpc error: code = NotFound desc = could not find container \"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554\": container with ID starting with 4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554 not found: ID does not exist"
Jul 16 21:52:57.385936 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385718    2551 scope.go:117] "RemoveContainer" containerID="7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"
Jul 16 21:52:57.386278 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.385975    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5\": container with ID starting with 7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5 not found: ID does not exist" containerID="7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"
Jul 16 21:52:57.386278 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.385994    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"} err="failed to get container status \"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5\": rpc error: code = NotFound desc = could not find container \"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5\": container with ID starting with 7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5 not found: ID does not exist"
Jul 16 21:52:57.386278 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386008    2551 scope.go:117] "RemoveContainer" containerID="22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"
Jul 16 21:52:57.386278 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.386237    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e\": container with ID starting with 22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e not found: ID does not exist" containerID="22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"
Jul 16 21:52:57.386423 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386274    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"} err="failed to get container status \"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e\": rpc error: code = NotFound desc = could not find container \"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e\": container with ID starting with 22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e not found: ID does not exist"
Jul 16 21:52:57.386423 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386292    2551 scope.go:117] "RemoveContainer" containerID="f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"
Jul 16 21:52:57.386516 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.386501    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc\": container with ID starting with f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc not found: ID does not exist" containerID="f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"
Jul 16 21:52:57.386554 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386520    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"} err="failed to get container status \"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc\": rpc error: code = NotFound desc = could not find container \"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc\": container with ID starting with f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc not found: ID does not exist"
Jul 16 21:52:57.386554 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386531    2551 scope.go:117] "RemoveContainer" containerID="cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"
Jul 16 21:52:57.386776 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.386752    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f\": container with ID starting with cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f not found: ID does not exist" containerID="cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"
Jul 16 21:52:57.386825 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386783    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"} err="failed to get container status \"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f\": rpc error: code = NotFound desc = could not find container \"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f\": container with ID starting with cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f not found: ID does not exist"
Jul 16 21:52:57.386825 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.386804    2551 scope.go:117] "RemoveContainer" containerID="5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"
Jul 16 21:52:57.387073 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.387054    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512\": container with ID starting with 5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512 not found: ID does not exist" containerID="5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"
Jul 16 21:52:57.387125 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387079    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"} err="failed to get container status \"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512\": rpc error: code = NotFound desc = could not find container \"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512\": container with ID starting with 5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512 not found: ID does not exist"
Jul 16 21:52:57.387125 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387093    2551 scope.go:117] "RemoveContainer" containerID="6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"
Jul 16 21:52:57.387450 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:52:57.387425    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b\": container with ID starting with 6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b not found: ID does not exist" containerID="6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"
Jul 16 21:52:57.387523 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387446    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"} err="failed to get container status \"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b\": rpc error: code = NotFound desc = could not find container \"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b\": container with ID starting with 6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b not found: ID does not exist"
Jul 16 21:52:57.387523 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387461    2551 scope.go:117] "RemoveContainer" containerID="4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"
Jul 16 21:52:57.387762 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387744    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554"} err="failed to get container status \"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554\": rpc error: code = NotFound desc = could not find container \"4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554\": container with ID starting with 4d5ba2eb3dd077f571910e7f984414c896a3967b8e735fd0fbb1ff1a162fe554 not found: ID does not exist"
Jul 16 21:52:57.387762 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.387761    2551 scope.go:117] "RemoveContainer" containerID="7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"
Jul 16 21:52:57.388032 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388013    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5"} err="failed to get container status \"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5\": rpc error: code = NotFound desc = could not find container \"7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5\": container with ID starting with 7e5302b4bdf36f479501b5a73e345fe06d7fa8a40d337b4c834ec731bd484ce5 not found: ID does not exist"
Jul 16 21:52:57.388079 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388033    2551 scope.go:117] "RemoveContainer" containerID="22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"
Jul 16 21:52:57.388295 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388273    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e"} err="failed to get container status \"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e\": rpc error: code = NotFound desc = could not find container \"22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e\": container with ID starting with 22915b9164446b07a87935615346ddd2a5aaaba2758ebec6dfc16b5973589a7e not found: ID does not exist"
Jul 16 21:52:57.388355 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388296    2551 scope.go:117] "RemoveContainer" containerID="f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"
Jul 16 21:52:57.388517 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388498    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc"} err="failed to get container status \"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc\": rpc error: code = NotFound desc = could not find container \"f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc\": container with ID starting with f5d5f63b7834a6caa3138307d04d277cadbfe276e54f9f6a79eee68111a348fc not found: ID does not exist"
Jul 16 21:52:57.388561 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388518    2551 scope.go:117] "RemoveContainer" containerID="cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"
Jul 16 21:52:57.388723 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388707    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f"} err="failed to get container status \"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f\": rpc error: code = NotFound desc = could not find container \"cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f\": container with ID starting with cbdaba009b01e95fcad71fecf545ddcbfc21269fe2b38ae2fc86d50e6b59e51f not found: ID does not exist"
Jul 16 21:52:57.388723 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388721    2551 scope.go:117] "RemoveContainer" containerID="5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"
Jul 16 21:52:57.388925 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388908    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512"} err="failed to get container status \"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512\": rpc error: code = NotFound desc = could not find container \"5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512\": container with ID starting with 5407630791d678bb6e992f1725bb93128d43b0eaf2cbd7b99ffff578144f2512 not found: ID does not exist"
Jul 16 21:52:57.388968 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.388925    2551 scope.go:117] "RemoveContainer" containerID="6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"
Jul 16 21:52:57.389114 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.389098    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b"} err="failed to get container status \"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b\": rpc error: code = NotFound desc = could not find container \"6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b\": container with ID starting with 6daa5c54e29740847e4a2f7e7940da4c24e5e393e992ce943871a7aad85e189b not found: ID does not exist"
Jul 16 21:52:57.390752 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.390738    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.393629 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393592    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 21:52:57.393629 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393608    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 21:52:57.393846 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393625    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 21:52:57.393846 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393657    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 21:52:57.393846 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393664    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k68q7\""
Jul 16 21:52:57.393846 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393676    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 21:52:57.394124 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393974    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 21:52:57.394124 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393974    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 21:52:57.394124 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.393976    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 21:52:57.399347 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.399319    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 21:52:57.408240 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.408215    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:57.512622 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512580    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512622 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512623    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512650    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512682    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512725    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-web-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512783    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-config-volume\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512816    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.512856 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512853    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.513064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512891    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-config-out\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.513064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512964    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.513064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.512990    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.513064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.513015    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.513064 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.513047    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnqqq\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-kube-api-access-gnqqq\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614447 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614366    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-config-out\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614447 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614410    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614447 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614430    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614457    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614481    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnqqq\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-kube-api-access-gnqqq\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614515    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614536    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614667    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614979 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614707    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614979 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614768    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-web-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614979 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614802    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-config-volume\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614979 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614831    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.614979 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.614885    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.615276 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.615228    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.615342 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.615319    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.615843 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.615814    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6956c2b-8332-40b1-aa07-e89e19d84181-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.617611 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.617148    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6956c2b-8332-40b1-aa07-e89e19d84181-config-out\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.617611 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.617526    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.617853 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.617829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.618872 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.618850    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.618964 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.618904    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-web-config\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.619591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.619131    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.619591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.619307    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.619591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.619360    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-config-volume\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.619591 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.619557    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f6956c2b-8332-40b1-aa07-e89e19d84181-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.623951 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.623929    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnqqq\" (UniqueName: \"kubernetes.io/projected/f6956c2b-8332-40b1-aa07-e89e19d84181-kube-api-access-gnqqq\") pod \"alertmanager-main-0\" (UID: \"f6956c2b-8332-40b1-aa07-e89e19d84181\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.700513 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.700475    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 21:52:57.842614 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:57.842584    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 21:52:57.846394 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:52:57.846369    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6956c2b_8332_40b1_aa07_e89e19d84181.slice/crio-210f8200cc3dc69dd82a72595785b1cbccfd1c721ab73752732d1bb32a4b572c WatchSource:0}: Error finding container 210f8200cc3dc69dd82a72595785b1cbccfd1c721ab73752732d1bb32a4b572c: Status 404 returned error can't find the container with id 210f8200cc3dc69dd82a72595785b1cbccfd1c721ab73752732d1bb32a4b572c
Jul 16 21:52:58.334132 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:58.334098    2551 generic.go:356] "Generic (PLEG): container finished" podID="f6956c2b-8332-40b1-aa07-e89e19d84181" containerID="17cafd3e451128e5262eb4a79cc510793a20b846b438f5ed68dbd1635239cc77" exitCode=0
Jul 16 21:52:58.334518 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:58.334180    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerDied","Data":"17cafd3e451128e5262eb4a79cc510793a20b846b438f5ed68dbd1635239cc77"}
Jul 16 21:52:58.334518 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:58.334246    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"210f8200cc3dc69dd82a72595785b1cbccfd1c721ab73752732d1bb32a4b572c"}
Jul 16 21:52:59.296321 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.296282    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67ab509-c56d-43db-8c72-de6c65afaebe" path="/var/lib/kubelet/pods/b67ab509-c56d-43db-8c72-de6c65afaebe/volumes"
Jul 16 21:52:59.342071 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342029    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"73dd6cab26c2e8d99bfda61028f90a3a7e3d8a98469b7355c837d28bb3882d41"}
Jul 16 21:52:59.342477 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342112    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"2f25db359ad391d9c61eee40bdf8edde73ad8cbcfea0131ce3620350a7248dcb"}
Jul 16 21:52:59.342477 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342157    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"df512838618605c6fc65592e33857fd87642859ecb63fcad168b3cf96b4bda38"}
Jul 16 21:52:59.342477 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342196    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"1d9d5cd22d1c98b7a6faa0207e830cd0e75ca3cd7e12ccbc8497b3da4e7ba020"}
Jul 16 21:52:59.342477 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342232    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"042122c67378a35afeaffc43dc2415d091df5d79726135cbc81e9767600fbff7"}
Jul 16 21:52:59.342477 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.342292    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f6956c2b-8332-40b1-aa07-e89e19d84181","Type":"ContainerStarted","Data":"d4ea10271a00d3fd674315dbd4204d4737bdc94d7300112ba9b942b04f9b5cce"}
Jul 16 21:52:59.594231 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.594122    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.583887979 podStartE2EDuration="2.594102614s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.214635ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:59.398049334 +0000 UTC m=+162.700222008" watchObservedRunningTime="2026-07-16 21:52:59.594102614 +0000 UTC m=+162.896275329"
Jul 16 21:52:59.595136 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595116    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:52:59.595463 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595426    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="prometheus" containerID="cri-o://3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd" gracePeriod=600
Jul 16 21:52:59.595463 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595448    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="thanos-sidecar" containerID="cri-o://0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1" gracePeriod=600
Jul 16 21:52:59.595463 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595462    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-web" containerID="cri-o://c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072" gracePeriod=600
Jul 16 21:52:59.595684 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595477    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="config-reloader" containerID="cri-o://d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5" gracePeriod=600
Jul 16 21:52:59.595684 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595490    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be" gracePeriod=600
Jul 16 21:52:59.595684 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:52:59.595439    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy" containerID="cri-o://b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065" gracePeriod=600
Jul 16 21:53:00.350311 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350277    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be" exitCode=0
Jul 16 21:53:00.350311 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350303    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065" exitCode=0
Jul 16 21:53:00.350311 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350309    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1" exitCode=0
Jul 16 21:53:00.350311 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350315    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5" exitCode=0
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350322    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd" exitCode=0
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350348    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"}
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350422    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"}
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350460    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"}
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350498    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"}
Jul 16 21:53:00.350708 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.350538    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"}
Jul 16 21:53:00.833768 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.833743    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:00.945887 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.945858    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.945896    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.945925    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.945943    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.945974    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946010    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx7t8\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946035 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946031    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946151    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946193    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946238    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946280    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946307    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946356    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946381    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946436 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946414    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946427    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946456    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946485    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946507    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946521    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946559    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946594    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1\") pod \"c0d9d265-dfbf-426e-910d-fdc95a951560\" (UID: \"c0d9d265-dfbf-426e-910d-fdc95a951560\") "
Jul 16 21:53:00.946802 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946763    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.947190 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946897    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:00.947190 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946920    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-metrics-client-ca\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:00.947190 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.946935    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:00.947190 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.947086    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.947762 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.947267    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:00.947762 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.947587    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.948666 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.947764    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.949076 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.949048    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:00.949904 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.949878    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.950003 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.949901    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:53:00.950003 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.949974    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.950202 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.950176    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out" (OuterVolumeSpecName: "config-out") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:00.950570 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.950533    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.950846 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.950817    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.950984 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.950872    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8" (OuterVolumeSpecName: "kube-api-access-rx7t8") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "kube-api-access-rx7t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:53:00.951516 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.951493    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.951795 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.951769    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.951795 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.951785    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.951939 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.951863    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config" (OuterVolumeSpecName: "config") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:00.962418 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:00.962395    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config" (OuterVolumeSpecName: "web-config") pod "c0d9d265-dfbf-426e-910d-fdc95a951560" (UID: "c0d9d265-dfbf-426e-910d-fdc95a951560"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:53:01.047681 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047645    2551 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-kube-rbac-proxy\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047681 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047675    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-db\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047681 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047684    2551 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047681 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047694    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx7t8\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-kube-api-access-rx7t8\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047702    2551 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-metrics-client-certs\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047711    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047720    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047729    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047740    2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0d9d265-dfbf-426e-910d-fdc95a951560-tls-assets\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047747    2551 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-grpc-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047759    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047776    2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0d9d265-dfbf-426e-910d-fdc95a951560-config-out\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047787    2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-web-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047794    2551 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047804    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0d9d265-dfbf-426e-910d-fdc95a951560-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047812    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.047948 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.047822    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c0d9d265-dfbf-426e-910d-fdc95a951560-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 21:53:01.357513 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.357478    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerID="c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072" exitCode=0
Jul 16 21:53:01.357897 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.357559    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.357897 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.357562    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"}
Jul 16 21:53:01.357897 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.357650    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c0d9d265-dfbf-426e-910d-fdc95a951560","Type":"ContainerDied","Data":"27e33f7df9c39f8acf23c86f671d00ab9f90b42d68a03ca775274478c777f1f2"}
Jul 16 21:53:01.357897 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.357701    2551 scope.go:117] "RemoveContainer" containerID="7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"
Jul 16 21:53:01.365774 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.365758    2551 scope.go:117] "RemoveContainer" containerID="b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"
Jul 16 21:53:01.377769 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.377746    2551 scope.go:117] "RemoveContainer" containerID="c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"
Jul 16 21:53:01.384672 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.384656    2551 scope.go:117] "RemoveContainer" containerID="0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"
Jul 16 21:53:01.392267 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.392227    2551 scope.go:117] "RemoveContainer" containerID="d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"
Jul 16 21:53:01.392792 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.392772    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:53:01.396284 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.396239    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:53:01.399585 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.399569    2551 scope.go:117] "RemoveContainer" containerID="3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"
Jul 16 21:53:01.406873 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.406854    2551 scope.go:117] "RemoveContainer" containerID="aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879"
Jul 16 21:53:01.413544 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.413528    2551 scope.go:117] "RemoveContainer" containerID="7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"
Jul 16 21:53:01.413807 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.413789    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be\": container with ID starting with 7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be not found: ID does not exist" containerID="7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"
Jul 16 21:53:01.413863 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.413817    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be"} err="failed to get container status \"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be\": rpc error: code = NotFound desc = could not find container \"7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be\": container with ID starting with 7fd9df527f25c058a0b502e55f9a5a3bf5cf62b4e861c7b7cd138c32bf10b2be not found: ID does not exist"
Jul 16 21:53:01.413863 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.413838    2551 scope.go:117] "RemoveContainer" containerID="b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"
Jul 16 21:53:01.414080 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.414065    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065\": container with ID starting with b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065 not found: ID does not exist" containerID="b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"
Jul 16 21:53:01.414120 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414086    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065"} err="failed to get container status \"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065\": rpc error: code = NotFound desc = could not find container \"b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065\": container with ID starting with b717bee1c34438c14bb2f49dd49ba3dd8e0e2262a46b3a0a1b37a8de11bc3065 not found: ID does not exist"
Jul 16 21:53:01.414120 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414101    2551 scope.go:117] "RemoveContainer" containerID="c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"
Jul 16 21:53:01.414320 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.414303    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072\": container with ID starting with c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072 not found: ID does not exist" containerID="c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"
Jul 16 21:53:01.414393 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414330    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072"} err="failed to get container status \"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072\": rpc error: code = NotFound desc = could not find container \"c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072\": container with ID starting with c3e9914a01e8c665c1bd12a78faa581388b3108f02484fcd53ff24480a435072 not found: ID does not exist"
Jul 16 21:53:01.414393 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414353    2551 scope.go:117] "RemoveContainer" containerID="0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"
Jul 16 21:53:01.414555 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.414536    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1\": container with ID starting with 0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1 not found: ID does not exist" containerID="0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"
Jul 16 21:53:01.414617 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414562    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1"} err="failed to get container status \"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1\": rpc error: code = NotFound desc = could not find container \"0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1\": container with ID starting with 0366029c3a27c3dc88928b1bb8e57d21d69722fb136ed9875ba5d0f2162d79c1 not found: ID does not exist"
Jul 16 21:53:01.414617 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414582    2551 scope.go:117] "RemoveContainer" containerID="d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"
Jul 16 21:53:01.414817 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.414800    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5\": container with ID starting with d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5 not found: ID does not exist" containerID="d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"
Jul 16 21:53:01.414862 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414820    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5"} err="failed to get container status \"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5\": rpc error: code = NotFound desc = could not find container \"d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5\": container with ID starting with d08174678a7651f4d6ee2ec5706c6687e51885ba177dae58390369478c6b56a5 not found: ID does not exist"
Jul 16 21:53:01.414862 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.414835    2551 scope.go:117] "RemoveContainer" containerID="3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"
Jul 16 21:53:01.415077 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.415056    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd\": container with ID starting with 3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd not found: ID does not exist" containerID="3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"
Jul 16 21:53:01.415137 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.415082    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd"} err="failed to get container status \"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd\": rpc error: code = NotFound desc = could not find container \"3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd\": container with ID starting with 3de74b9fd59896fe36fed431bfd1c262e47c3a485739a997885336193b723ccd not found: ID does not exist"
Jul 16 21:53:01.415137 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.415098    2551 scope.go:117] "RemoveContainer" containerID="aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879"
Jul 16 21:53:01.415408 ip-10-0-138-41 kubenswrapper[2551]: E0716 21:53:01.415376    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879\": container with ID starting with aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879 not found: ID does not exist" containerID="aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879"
Jul 16 21:53:01.415463 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.415414    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879"} err="failed to get container status \"aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879\": rpc error: code = NotFound desc = could not find container \"aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879\": container with ID starting with aabcc75845a5f963b110b68bab8a6cef1a10a63c9fc0ecb7ce184caac7845879 not found: ID does not exist"
Jul 16 21:53:01.431048 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431003    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:53:01.431556 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431538    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="prometheus"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431558    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="prometheus"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431568    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-web"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431574    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-web"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431585    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-thanos"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431590    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-thanos"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431598    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="thanos-sidecar"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431604    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="thanos-sidecar"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431612    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy"
Jul 16 21:53:01.431619 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431617    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431630    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="init-config-reloader"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431635    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="init-config-reloader"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431643    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="config-reloader"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431648    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="config-reloader"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431698    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-thanos"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431708    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="prometheus"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431715    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="config-reloader"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431720    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="thanos-sidecar"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431725    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy"
Jul 16 21:53:01.432006 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.431732    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" containerName="kube-rbac-proxy-web"
Jul 16 21:53:01.436562 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.436544    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.442864 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.442827    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 21:53:01.443025 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.442833    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 21:53:01.443353 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443339    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 21:53:01.443457 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443348    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e086k6jp8d013\""
Jul 16 21:53:01.443457 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443364    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 21:53:01.443457 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443348    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xjrcq\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443952    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443962    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443991    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.444004    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.444036    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443992    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.443993    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 21:53:01.444147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.444106    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 21:53:01.446458 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.446442    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 21:53:01.447599 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.447585    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 21:53:01.452712 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.452697    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 21:53:01.465949 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.465928    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:53:01.550807 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550772    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.550946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550814    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.550946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550842    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.550946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550868    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.550946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550916    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551088 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550958    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551088 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.550984    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551088 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551014    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551088 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551069    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mnn\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-kube-api-access-q6mnn\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551092    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551121    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551148    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551168    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551199 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551184    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551216    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551242    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551311    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551332    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551368    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.551383 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.551382    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652721 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652633    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652721 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652671    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652721 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652803    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652854    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652893    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.652946 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652921    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652947    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.652972    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653004    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653042    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653080    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653147 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653128    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mnn\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-kube-api-access-q6mnn\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653161    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653183    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653226    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653289    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653319    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653348    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653480 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653379    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653813 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.653813 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.653798    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.654083 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.654054    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.654412 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.654386    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.654489 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.654413    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.655177 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.654881    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.655844 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.655814    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.655958 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.655868    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.655958 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.655952    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.656533 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.656465    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.657005 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.656964    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.657324 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.657212    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.657702 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.657675    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.657792 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.657739    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.658323 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.658303    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f818707-5e99-4c32-a111-4322be57f513-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.658531 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.658512    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.658768 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.658741    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.659056 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.659037    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f818707-5e99-4c32-a111-4322be57f513-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.659713 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.659695    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f818707-5e99-4c32-a111-4322be57f513-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.669794 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.669772    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mnn\" (UniqueName: \"kubernetes.io/projected/7f818707-5e99-4c32-a111-4322be57f513-kube-api-access-q6mnn\") pod \"prometheus-k8s-0\" (UID: \"7f818707-5e99-4c32-a111-4322be57f513\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.746684 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.746643    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:53:01.896883 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:01.896814    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 21:53:01.900514 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:53:01.900492    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f818707_5e99_4c32_a111_4322be57f513.slice/crio-28957def7f3d3db16bffd51455407b9c3f7b23a0401e80fb90675dcb2b31bf65 WatchSource:0}: Error finding container 28957def7f3d3db16bffd51455407b9c3f7b23a0401e80fb90675dcb2b31bf65: Status 404 returned error can't find the container with id 28957def7f3d3db16bffd51455407b9c3f7b23a0401e80fb90675dcb2b31bf65
Jul 16 21:53:02.363021 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:02.362985    2551 generic.go:356] "Generic (PLEG): container finished" podID="7f818707-5e99-4c32-a111-4322be57f513" containerID="8d2b384deeabf6e8028131b167480a385488fa241d9316a1f0416b4f1f76c500" exitCode=0
Jul 16 21:53:02.363476 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:02.363072    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerDied","Data":"8d2b384deeabf6e8028131b167480a385488fa241d9316a1f0416b4f1f76c500"}
Jul 16 21:53:02.363476 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:02.363139    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"28957def7f3d3db16bffd51455407b9c3f7b23a0401e80fb90675dcb2b31bf65"}
Jul 16 21:53:03.296110 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.296071    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d9d265-dfbf-426e-910d-fdc95a951560" path="/var/lib/kubelet/pods/c0d9d265-dfbf-426e-910d-fdc95a951560/volumes"
Jul 16 21:53:03.371842 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.371803    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"79eb007179f0cf35cd501b1c38470a9a2bb3ad45c499d9d686c57e1c05cc3949"}
Jul 16 21:53:03.372210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.371896    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"55ba7f2c40c280a5e39439bfa1a3578c1cc2de8c48830d141bad8ffb13c507bb"}
Jul 16 21:53:03.372210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.371928    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"64eaa5dde32e54325622d10f56603523912a6e40e77d10fd3b39af09921d8d38"}
Jul 16 21:53:03.372210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.371956    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"d9eab6c59ab5dd503d07b5284945097838348ee263dcd909f5ca8ce807d10e0e"}
Jul 16 21:53:03.372210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.371995    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"3af54cc8325aef5341502c868e7b62d4b925697d98c4a8e359f82d73f603c742"}
Jul 16 21:53:03.372210 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.372020    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f818707-5e99-4c32-a111-4322be57f513","Type":"ContainerStarted","Data":"e39763069df644328c29a835a8ea685b79e3e74c740faf1c5e3c7b78ffc9f99c"}
Jul 16 21:53:03.410229 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:03.410167    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.390161667 podStartE2EDuration="2.410144082s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.982415ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:03.407331177 +0000 UTC m=+166.709503870" watchObservedRunningTime="2026-07-16 21:53:03.410144082 +0000 UTC m=+166.712316756"
Jul 16 21:53:06.747578 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:53:06.747540    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:54:01.747246 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:54:01.747206    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:54:01.762583 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:54:01.762557    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:54:02.627178 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:54:02.627146    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 21:55:17.115096 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:55:17.115069    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 21:55:17.115656 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:55:17.115103    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 21:55:17.124349 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:55:17.124331    2551 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 21:58:45.030458 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.030425    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-gdqt7"]
Jul 16 21:58:45.033458 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.033441    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.037434 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.037407    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 16 21:58:45.037692 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.037676    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-8867j\""
Jul 16 21:58:45.037998 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.037984    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 16 21:58:45.038036 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.037996    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 16 21:58:45.043788 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.043762    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-gdqt7"]
Jul 16 21:58:45.093807 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.093766    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.093977 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.093844    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8j7x\" (UniqueName: \"kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.194809 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.194752    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.194988 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.194854    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8j7x\" (UniqueName: \"kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.195148 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.195124    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.205157 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.205129    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8j7x\" (UniqueName: \"kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x\") pod \"s3-init-gdqt7\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") " pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.357188 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.357103    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gdqt7"
Jul 16 21:58:45.483550 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.483527    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-gdqt7"]
Jul 16 21:58:45.485665 ip-10-0-138-41 kubenswrapper[2551]: W0716 21:58:45.485631    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87c1b0e7_3cd5_4c9e_8c49_8759c342bfb5.slice/crio-5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab WatchSource:0}: Error finding container 5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab: Status 404 returned error can't find the container with id 5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab
Jul 16 21:58:45.487564 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.487546    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:58:45.745049 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:45.745015    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerStarted","Data":"5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab"}
Jul 16 21:58:49.767004 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:49.766967    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerStarted","Data":"76e1442cffe9a1c2d51d0c8b0f44e8df6372be173dc12da392c3fd9945a4ab87"}
Jul 16 21:58:51.775868 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:51.775839    2551 generic.go:356] "Generic (PLEG): container finished" podID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerID="76e1442cffe9a1c2d51d0c8b0f44e8df6372be173dc12da392c3fd9945a4ab87" exitCode=0
Jul 16 21:58:51.776230 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:51.775909    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerDied","Data":"76e1442cffe9a1c2d51d0c8b0f44e8df6372be173dc12da392c3fd9945a4ab87"}
Jul 16 21:58:56.802722 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:56.802631    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerStarted","Data":"e33bb0b0b223829e4d5c6d33c1b89765090b9057c3389d0a239ea671ea4f673d"}
Jul 16 21:58:56.821576 ip-10-0-138-41 kubenswrapper[2551]: I0716 21:58:56.821525    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-gdqt7" podStartSLOduration=1.8522253929999999 podStartE2EDuration="11.821509684s" totalImagesPullingTime="8.374028522s" totalInitContainerRuntime="1.595255769s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:56.820383573 +0000 UTC m=+520.122556256" watchObservedRunningTime="2026-07-16 21:58:56.821509684 +0000 UTC m=+520.123682356"
Jul 16 22:00:02.072639 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:02.072605    2551 generic.go:356] "Generic (PLEG): container finished" podID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerID="e33bb0b0b223829e4d5c6d33c1b89765090b9057c3389d0a239ea671ea4f673d" exitCode=0
Jul 16 22:00:02.073062 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:02.072688    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerDied","Data":"e33bb0b0b223829e4d5c6d33c1b89765090b9057c3389d0a239ea671ea4f673d"}
Jul 16 22:00:03.206448 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.206424    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gdqt7"
Jul 16 22:00:03.349841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.349756    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache\") pod \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") "
Jul 16 22:00:03.349841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.349839    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8j7x\" (UniqueName: \"kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x\") pod \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\" (UID: \"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5\") "
Jul 16 22:00:03.352007 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.351982    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x" (OuterVolumeSpecName: "kube-api-access-b8j7x") pod "87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" (UID: "87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5"). InnerVolumeSpecName "kube-api-access-b8j7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:00:03.368280 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.368230    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" (UID: "87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:00:03.451215 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.451179    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-model-cache\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:03.451215 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:03.451206    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8j7x\" (UniqueName: \"kubernetes.io/projected/87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5-kube-api-access-b8j7x\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:04.087496 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:04.087466    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gdqt7"
Jul 16 22:00:04.087496 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:04.087481    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gdqt7" event={"ID":"87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5","Type":"ContainerDied","Data":"5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab"}
Jul 16 22:00:04.087757 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:04.087514    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b16b0a31e2c73bc14dbed7e80075907b29563cfdfdcf6c2a67590dc51c241ab"
Jul 16 22:00:14.871593 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871558    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-cggml"]
Jul 16 22:00:14.872047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871884    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerName="s3-init"
Jul 16 22:00:14.872047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871895    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerName="s3-init"
Jul 16 22:00:14.872047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871921    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerName="download-hf-model"
Jul 16 22:00:14.872047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871927    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerName="download-hf-model"
Jul 16 22:00:14.872047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.871982    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5" containerName="s3-init"
Jul 16 22:00:14.875002 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.874986    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:14.877751 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.877733    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 16 22:00:14.878781 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.878766    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-8867j\""
Jul 16 22:00:14.878841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.878766    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 16 22:00:14.878841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.878767    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 16 22:00:14.883368 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:14.883348    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-cggml"]
Jul 16 22:00:15.048332 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.048288    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mp82\" (UniqueName: \"kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.048507 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.048353    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.149803 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.149718    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mp82\" (UniqueName: \"kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.149803 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.149764    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.150106 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.150090    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.164365 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.164340    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mp82\" (UniqueName: \"kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82\") pod \"s3-tls-init-custom-cggml\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") " pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.204188 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.204161    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:15.346919 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:15.346896    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-cggml"]
Jul 16 22:00:15.348961 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:00:15.348934    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod239c97bf_50cf_4971_87e8_dff44c323ed6.slice/crio-748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7 WatchSource:0}: Error finding container 748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7: Status 404 returned error can't find the container with id 748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7
Jul 16 22:00:16.138194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:16.138157    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerStarted","Data":"2e3491ae01c97ede96a66181c82ee97ce678fcde2de8ad5b64aa68b0effd8ff3"}
Jul 16 22:00:16.138194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:16.138202    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerStarted","Data":"748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7"}
Jul 16 22:00:17.143535 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:17.143443    2551 generic.go:356] "Generic (PLEG): container finished" podID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerID="2e3491ae01c97ede96a66181c82ee97ce678fcde2de8ad5b64aa68b0effd8ff3" exitCode=0
Jul 16 22:00:17.143535 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:17.143495    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:00:17.144061 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:17.143524    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerDied","Data":"2e3491ae01c97ede96a66181c82ee97ce678fcde2de8ad5b64aa68b0effd8ff3"}
Jul 16 22:00:17.145548 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:17.145527    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:00:17.159598 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:17.159577    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 16 22:00:18.149936 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:18.149892    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerStarted","Data":"74815488d14b12c8bc50ddc61bc1f4bfab717600c92dd48e0c2619d7787f30df"}
Jul 16 22:00:18.175754 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:18.175685    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-cggml" podStartSLOduration=2.797373944 podStartE2EDuration="4.175666175s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.378292231s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:18.175448957 +0000 UTC m=+601.477621632" watchObservedRunningTime="2026-07-16 22:00:18.175666175 +0000 UTC m=+601.477838849"
Jul 16 22:00:24.176409 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:24.176371    2551 generic.go:356] "Generic (PLEG): container finished" podID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerID="74815488d14b12c8bc50ddc61bc1f4bfab717600c92dd48e0c2619d7787f30df" exitCode=0
Jul 16 22:00:24.176409 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:24.176405    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerDied","Data":"74815488d14b12c8bc50ddc61bc1f4bfab717600c92dd48e0c2619d7787f30df"}
Jul 16 22:00:25.327217 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.327188    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:25.439961 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.439873    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mp82\" (UniqueName: \"kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82\") pod \"239c97bf-50cf-4971-87e8-dff44c323ed6\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") "
Jul 16 22:00:25.440119 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.439971    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache\") pod \"239c97bf-50cf-4971-87e8-dff44c323ed6\" (UID: \"239c97bf-50cf-4971-87e8-dff44c323ed6\") "
Jul 16 22:00:25.442091 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.442062    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82" (OuterVolumeSpecName: "kube-api-access-4mp82") pod "239c97bf-50cf-4971-87e8-dff44c323ed6" (UID: "239c97bf-50cf-4971-87e8-dff44c323ed6"). InnerVolumeSpecName "kube-api-access-4mp82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:00:25.462671 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.462617    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache" (OuterVolumeSpecName: "model-cache") pod "239c97bf-50cf-4971-87e8-dff44c323ed6" (UID: "239c97bf-50cf-4971-87e8-dff44c323ed6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:00:25.540781 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.540744    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/239c97bf-50cf-4971-87e8-dff44c323ed6-model-cache\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:25.540781 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:25.540777    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mp82\" (UniqueName: \"kubernetes.io/projected/239c97bf-50cf-4971-87e8-dff44c323ed6-kube-api-access-4mp82\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:26.186085 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:26.186055    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-cggml"
Jul 16 22:00:26.186085 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:26.186071    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-cggml" event={"ID":"239c97bf-50cf-4971-87e8-dff44c323ed6","Type":"ContainerDied","Data":"748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7"}
Jul 16 22:00:26.186295 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:26.186104    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="748f965042ed8e5c9b0e57b1d6707f508e988b651dd9cb89db971f2fcffe80d7"
Jul 16 22:00:29.028837 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.028803    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-wd6nx"]
Jul 16 22:00:29.029238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.029194    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerName="download-hf-model"
Jul 16 22:00:29.029238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.029209    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerName="download-hf-model"
Jul 16 22:00:29.029238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.029227    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerName="s3-tls-init-custom"
Jul 16 22:00:29.029238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.029232    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerName="s3-tls-init-custom"
Jul 16 22:00:29.029448 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.029327    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="239c97bf-50cf-4971-87e8-dff44c323ed6" containerName="s3-tls-init-custom"
Jul 16 22:00:29.033211 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.033195    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.036314 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.036284    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 16 22:00:29.036448 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.036331    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 16 22:00:29.036577 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.036562    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-8867j\""
Jul 16 22:00:29.036659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.036610    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 16 22:00:29.042789 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.042766    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-wd6nx"]
Jul 16 22:00:29.072060 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.072033    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.072235 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.072092    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56d5q\" (UniqueName: \"kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.173241 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.173203    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56d5q\" (UniqueName: \"kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.173439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.173317    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.173770 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.173751    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.182796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.182765    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56d5q\" (UniqueName: \"kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q\") pod \"s3-tls-init-serving-wd6nx\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") " pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.355590 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.355508    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:29.490117 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:29.490069    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-wd6nx"]
Jul 16 22:00:29.492451 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:00:29.492423    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod907f1a93_e828_48f4_ab5c_e2b04c2e4684.slice/crio-25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd WatchSource:0}: Error finding container 25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd: Status 404 returned error can't find the container with id 25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd
Jul 16 22:00:30.207171 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:30.207133    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerStarted","Data":"aa8595474f1b23d119fe8f63526bffaf560cc04b17178831c6e997671e0ef9db"}
Jul 16 22:00:30.207571 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:30.207178    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerStarted","Data":"25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd"}
Jul 16 22:00:31.216688 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:31.216649    2551 generic.go:356] "Generic (PLEG): container finished" podID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerID="aa8595474f1b23d119fe8f63526bffaf560cc04b17178831c6e997671e0ef9db" exitCode=0
Jul 16 22:00:31.217144 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:31.216741    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerDied","Data":"aa8595474f1b23d119fe8f63526bffaf560cc04b17178831c6e997671e0ef9db"}
Jul 16 22:00:32.222488 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:32.222451    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerStarted","Data":"9b882763204f0a9bc2e0b05a9004af018dceb8edea84d7a841819076afc702ac"}
Jul 16 22:00:32.242699 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:32.242650    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-wd6nx" podStartSLOduration=2.197430033 podStartE2EDuration="3.242636006s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.045205973s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:32.240954415 +0000 UTC m=+615.543127089" watchObservedRunningTime="2026-07-16 22:00:32.242636006 +0000 UTC m=+615.544808679"
Jul 16 22:00:37.247802 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:37.247771    2551 generic.go:356] "Generic (PLEG): container finished" podID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerID="9b882763204f0a9bc2e0b05a9004af018dceb8edea84d7a841819076afc702ac" exitCode=0
Jul 16 22:00:37.248152 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:37.247853    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerDied","Data":"9b882763204f0a9bc2e0b05a9004af018dceb8edea84d7a841819076afc702ac"}
Jul 16 22:00:38.389047 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.389021    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:38.462577 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.462542    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache\") pod \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") "
Jul 16 22:00:38.462577 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.462578    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56d5q\" (UniqueName: \"kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q\") pod \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\" (UID: \"907f1a93-e828-48f4-ab5c-e2b04c2e4684\") "
Jul 16 22:00:38.464779 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.464757    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q" (OuterVolumeSpecName: "kube-api-access-56d5q") pod "907f1a93-e828-48f4-ab5c-e2b04c2e4684" (UID: "907f1a93-e828-48f4-ab5c-e2b04c2e4684"). InnerVolumeSpecName "kube-api-access-56d5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:00:38.484896 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.484865    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache" (OuterVolumeSpecName: "model-cache") pod "907f1a93-e828-48f4-ab5c-e2b04c2e4684" (UID: "907f1a93-e828-48f4-ab5c-e2b04c2e4684"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:00:38.563442 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.563362    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/907f1a93-e828-48f4-ab5c-e2b04c2e4684-model-cache\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:38.563442 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:38.563392    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56d5q\" (UniqueName: \"kubernetes.io/projected/907f1a93-e828-48f4-ab5c-e2b04c2e4684-kube-api-access-56d5q\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:00:39.257578 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:39.257509    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wd6nx"
Jul 16 22:00:39.257703 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:39.257508    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wd6nx" event={"ID":"907f1a93-e828-48f4-ab5c-e2b04c2e4684","Type":"ContainerDied","Data":"25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd"}
Jul 16 22:00:39.257703 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:39.257621    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25011b5685d0f0076f8aea019b2b899b17ce73cd2440596fbb79dd771b0faedd"
Jul 16 22:00:51.683329 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683295    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:00:51.683796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683649    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerName="download-hf-model"
Jul 16 22:00:51.683796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683660    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerName="download-hf-model"
Jul 16 22:00:51.683796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683682    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerName="s3-tls-init-serving"
Jul 16 22:00:51.683796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683689    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerName="s3-tls-init-serving"
Jul 16 22:00:51.683796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.683758    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="907f1a93-e828-48f4-ab5c-e2b04c2e4684" containerName="s3-tls-init-serving"
Jul 16 22:00:51.687236 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.687218    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.690967 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.690942    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 16 22:00:51.691075 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.690989    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 16 22:00:51.691156 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.691143    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lxc2m\""
Jul 16 22:00:51.692139 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.692121    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 16 22:00:51.692139 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.692133    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 16 22:00:51.692286 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.692160    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 16 22:00:51.692286 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.692219    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 16 22:00:51.709176 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.709150    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:00:51.775682 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.775653    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.775682 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.775684    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.775868 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.775705    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvsnh\" (UniqueName: \"kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.775868 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.775760    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.775868 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.775817    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.876684 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.876650    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.876858 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.876733    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.876858 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.876751    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.876858 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.876770    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvsnh\" (UniqueName: \"kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.876858 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.876790    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.877306 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.877280    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.877460 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.877430    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.877527 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.877510    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.879225 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.879204    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.886762 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.886737    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvsnh\" (UniqueName: \"kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh\") pod \"isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:51.997321 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:51.997223    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:00:52.153917 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:52.153887    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:00:52.155266 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:00:52.155229    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf303239_7ee1_47e5_a191_b778acd68a10.slice/crio-de3ba9128c2037bf127c5e1b60a3700c9f2a1c903fcacb243f40981da04643ad WatchSource:0}: Error finding container de3ba9128c2037bf127c5e1b60a3700c9f2a1c903fcacb243f40981da04643ad: Status 404 returned error can't find the container with id de3ba9128c2037bf127c5e1b60a3700c9f2a1c903fcacb243f40981da04643ad
Jul 16 22:00:52.311993 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:52.311899    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerStarted","Data":"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"}
Jul 16 22:00:52.311993 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:52.311976    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerStarted","Data":"de3ba9128c2037bf127c5e1b60a3700c9f2a1c903fcacb243f40981da04643ad"}
Jul 16 22:00:56.331863 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:56.331829    2551 generic.go:356] "Generic (PLEG): container finished" podID="cf303239-7ee1-47e5-a191-b778acd68a10" containerID="f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a" exitCode=0
Jul 16 22:00:56.332339 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:00:56.331870    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerDied","Data":"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"}
Jul 16 22:01:10.411662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:10.411572    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerStarted","Data":"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"}
Jul 16 22:01:12.422936 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:12.422900    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerStarted","Data":"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"}
Jul 16 22:01:15.438395 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:15.438354    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerStarted","Data":"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"}
Jul 16 22:01:15.438791 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:15.438504    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:01:15.438791 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:15.438562    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:01:15.439973 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:15.439939    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:15.463491 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:15.463443    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podStartSLOduration=1.958291872 podStartE2EDuration="24.463425926s" totalImagesPullingTime="18.563960854s" totalInitContainerRuntime="3.9411732s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:01:15.46144869 +0000 UTC m=+658.763621382" watchObservedRunningTime="2026-07-16 22:01:15.463425926 +0000 UTC m=+658.765598597"
Jul 16 22:01:16.442875 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:16.442835    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:01:16.443334 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:16.442991    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:16.443889 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:16.443865    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:17.446904 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:17.446860    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:17.447367 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:17.447321    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:17.451973 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:17.451951    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:01:18.450954 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:18.450913    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:18.451397 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:18.451226    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:28.451732 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:28.451682    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:28.452227 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:28.452091    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:38.451873 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:38.451831    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:38.452367 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:38.452300    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:48.451356 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:48.451307    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:48.451797 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:48.451773    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:01:58.450900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:58.450846    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:01:58.451322 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:01:58.451219    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:08.450996 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:08.450943    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:02:08.451560 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:08.451442    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:18.451532 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:18.451435    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:02:18.451976 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:18.451913    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:28.452180 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:28.452145    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:02:28.452658 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:28.452340    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:02:36.675037 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.675005    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:02:36.675566 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.675361    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" containerID="cri-o://88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf" gracePeriod=30
Jul 16 22:02:36.675566 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.675374    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" containerID="cri-o://d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a" gracePeriod=30
Jul 16 22:02:36.675566 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.675385    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" containerID="cri-o://793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f" gracePeriod=30
Jul 16 22:02:36.793587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.793560    2551 generic.go:356] "Generic (PLEG): container finished" podID="cf303239-7ee1-47e5-a191-b778acd68a10" containerID="793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f" exitCode=2
Jul 16 22:02:36.793667 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.793628    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerDied","Data":"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"}
Jul 16 22:02:36.821247 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.821215    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:02:36.825490 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.825472    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:36.830373 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.830353    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 16 22:02:36.830493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.830386    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 16 22:02:36.839463 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:36.839440    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:02:37.009210 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.009113    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.009210 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.009169    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.009432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.009306    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp6hm\" (UniqueName: \"kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.009432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.009339    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.009432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.009420    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.110451 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110419    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.110649 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110508    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.110649 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110548    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.110649 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110603    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp6hm\" (UniqueName: \"kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.110649 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110629    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.111010 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.110979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.111288 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.111241    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.111350 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.111314    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.113031 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.113009    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.122869 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.122836    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp6hm\" (UniqueName: \"kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm\") pod \"isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.135721 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.135697    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:37.286826 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.286795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:02:37.287929 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:02:37.287901    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9095d34f_73af_4ff2_8db2_fd05434b67f5.slice/crio-86038727f549a5c28b19d89c3f5692c1022df5dbee3b10c1e0790b895beb4418 WatchSource:0}: Error finding container 86038727f549a5c28b19d89c3f5692c1022df5dbee3b10c1e0790b895beb4418: Status 404 returned error can't find the container with id 86038727f549a5c28b19d89c3f5692c1022df5dbee3b10c1e0790b895beb4418
Jul 16 22:02:37.447487 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.447448    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:02:37.804055 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.804020    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerStarted","Data":"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"}
Jul 16 22:02:37.804432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:37.804087    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerStarted","Data":"86038727f549a5c28b19d89c3f5692c1022df5dbee3b10c1e0790b895beb4418"}
Jul 16 22:02:38.451374 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:38.451321    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:02:38.451688 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:38.451660    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:41.824563 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:41.824524    2551 generic.go:356] "Generic (PLEG): container finished" podID="cf303239-7ee1-47e5-a191-b778acd68a10" containerID="88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf" exitCode=0
Jul 16 22:02:41.825030 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:41.824599    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerDied","Data":"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"}
Jul 16 22:02:41.826101 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:41.826081    2551 generic.go:356] "Generic (PLEG): container finished" podID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerID="1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4" exitCode=0
Jul 16 22:02:41.826189 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:41.826111    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerDied","Data":"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"}
Jul 16 22:02:42.447266 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.447208    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:02:42.833317 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.833220    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerStarted","Data":"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"}
Jul 16 22:02:42.833317 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.833306    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerStarted","Data":"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"}
Jul 16 22:02:42.833728 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.833334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerStarted","Data":"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"}
Jul 16 22:02:42.833728 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.833529    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:42.858760 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:42.858716    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podStartSLOduration=2.8562693919999997 podStartE2EDuration="6.858701194s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.002431802s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:02:42.855843298 +0000 UTC m=+746.158015994" watchObservedRunningTime="2026-07-16 22:02:42.858701194 +0000 UTC m=+746.160873921"
Jul 16 22:02:43.838210 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:43.838176    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:43.838694 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:43.838273    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:43.839567 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:43.839540    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:02:43.840218 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:43.840188    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:44.842624 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:44.842586    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:02:44.843009 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:44.842981    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:47.447089 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:47.447034    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:02:47.447643 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:47.447160    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:02:48.451455 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:48.451411    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:02:48.451932 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:48.451778    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:49.846625 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:49.846596    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:02:49.847140 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:49.847114    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:02:49.847389 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:49.847367    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:52.447832 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:52.447787    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:02:57.447713 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:57.447663    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:02:58.450923 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:58.450882    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 16 22:02:58.451402 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:58.451089    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:02:58.451402 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:58.451169    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:02:58.451402 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:58.451295    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:02:59.847865 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:59.847824    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:02:59.848367 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:02:59.848268    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:02.447155 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:02.447109    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 16 22:03:06.862227 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.862201    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:03:06.942596 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.942511    2551 generic.go:356] "Generic (PLEG): container finished" podID="cf303239-7ee1-47e5-a191-b778acd68a10" containerID="d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a" exitCode=0
Jul 16 22:03:06.942596 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.942556    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerDied","Data":"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"}
Jul 16 22:03:06.942596 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.942572    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"
Jul 16 22:03:06.942818 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.942609    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65" event={"ID":"cf303239-7ee1-47e5-a191-b778acd68a10","Type":"ContainerDied","Data":"de3ba9128c2037bf127c5e1b60a3700c9f2a1c903fcacb243f40981da04643ad"}
Jul 16 22:03:06.942818 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.942648    2551 scope.go:117] "RemoveContainer" containerID="d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"
Jul 16 22:03:06.951038 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.951020    2551 scope.go:117] "RemoveContainer" containerID="793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"
Jul 16 22:03:06.958837 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.958818    2551 scope.go:117] "RemoveContainer" containerID="88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"
Jul 16 22:03:06.966225 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.966207    2551 scope.go:117] "RemoveContainer" containerID="f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"
Jul 16 22:03:06.973659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.973641    2551 scope.go:117] "RemoveContainer" containerID="d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"
Jul 16 22:03:06.973935 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:03:06.973909    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a\": container with ID starting with d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a not found: ID does not exist" containerID="d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"
Jul 16 22:03:06.974034 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.973938    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a"} err="failed to get container status \"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a\": rpc error: code = NotFound desc = could not find container \"d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a\": container with ID starting with d338cf24784837f1bd2cd249b6091f667623945f2e7bdf9ba7d1cb5ea333aa2a not found: ID does not exist"
Jul 16 22:03:06.974034 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.973957    2551 scope.go:117] "RemoveContainer" containerID="793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"
Jul 16 22:03:06.974198 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:03:06.974180    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f\": container with ID starting with 793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f not found: ID does not exist" containerID="793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"
Jul 16 22:03:06.974246 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.974204    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f"} err="failed to get container status \"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f\": rpc error: code = NotFound desc = could not find container \"793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f\": container with ID starting with 793f1eab5ed9e55e728332bb18bced40f1da85afbec2b0e7ca1271cf26c16a4f not found: ID does not exist"
Jul 16 22:03:06.974246 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.974221    2551 scope.go:117] "RemoveContainer" containerID="88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"
Jul 16 22:03:06.974522 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:03:06.974506    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf\": container with ID starting with 88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf not found: ID does not exist" containerID="88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"
Jul 16 22:03:06.974570 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.974525    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf"} err="failed to get container status \"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf\": rpc error: code = NotFound desc = could not find container \"88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf\": container with ID starting with 88e01b71a510749bb71aa2f6f0c687ac54375ac9cc0ba0d540b493a6beed2dbf not found: ID does not exist"
Jul 16 22:03:06.974570 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.974537    2551 scope.go:117] "RemoveContainer" containerID="f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"
Jul 16 22:03:06.974747 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:03:06.974733    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a\": container with ID starting with f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a not found: ID does not exist" containerID="f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"
Jul 16 22:03:06.974786 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.974749    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a"} err="failed to get container status \"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a\": rpc error: code = NotFound desc = could not find container \"f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a\": container with ID starting with f3d4e2544d1e811b6013a0d7156361b5f8689a46ef1af492174b127a3a41792a not found: ID does not exist"
Jul 16 22:03:06.980044 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980026    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location\") pod \"cf303239-7ee1-47e5-a191-b778acd68a10\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") "
Jul 16 22:03:06.980138 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980126    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls\") pod \"cf303239-7ee1-47e5-a191-b778acd68a10\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") "
Jul 16 22:03:06.980207 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980183    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvsnh\" (UniqueName: \"kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh\") pod \"cf303239-7ee1-47e5-a191-b778acd68a10\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") "
Jul 16 22:03:06.980286 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980212    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert\") pod \"cf303239-7ee1-47e5-a191-b778acd68a10\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") "
Jul 16 22:03:06.980286 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980244    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cf303239-7ee1-47e5-a191-b778acd68a10" (UID: "cf303239-7ee1-47e5-a191-b778acd68a10"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:03:06.980392 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980285    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config\") pod \"cf303239-7ee1-47e5-a191-b778acd68a10\" (UID: \"cf303239-7ee1-47e5-a191-b778acd68a10\") "
Jul 16 22:03:06.980546 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980526    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf303239-7ee1-47e5-a191-b778acd68a10-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:03:06.980654 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980569    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "cf303239-7ee1-47e5-a191-b778acd68a10" (UID: "cf303239-7ee1-47e5-a191-b778acd68a10"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:03:06.980721 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.980695    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cf303239-7ee1-47e5-a191-b778acd68a10" (UID: "cf303239-7ee1-47e5-a191-b778acd68a10"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:03:06.982313 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.982276    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cf303239-7ee1-47e5-a191-b778acd68a10" (UID: "cf303239-7ee1-47e5-a191-b778acd68a10"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:03:06.982313 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:06.982245    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh" (OuterVolumeSpecName: "kube-api-access-rvsnh") pod "cf303239-7ee1-47e5-a191-b778acd68a10" (UID: "cf303239-7ee1-47e5-a191-b778acd68a10"). InnerVolumeSpecName "kube-api-access-rvsnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:03:07.081659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.081624    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf303239-7ee1-47e5-a191-b778acd68a10-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:03:07.081659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.081651    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvsnh\" (UniqueName: \"kubernetes.io/projected/cf303239-7ee1-47e5-a191-b778acd68a10-kube-api-access-rvsnh\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:03:07.081659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.081664    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:03:07.081897 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.081675    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf303239-7ee1-47e5-a191-b778acd68a10-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:03:07.266592 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.266531    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:03:07.268384 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.268361    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5f987ddb5f-mkw65"]
Jul 16 22:03:07.296192 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:07.296162    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" path="/var/lib/kubelet/pods/cf303239-7ee1-47e5-a191-b778acd68a10/volumes"
Jul 16 22:03:09.847561 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:09.847515    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:03:09.847972 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:09.847872    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:19.847203 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:19.847156    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:03:19.847673 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:19.847652    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:29.847924 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:29.847881    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:03:29.848419 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:29.848398    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:39.847823 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:39.847732    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:03:39.848314 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:39.848289    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:49.847439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:49.847396    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:03:49.847927 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:49.847862    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:03:59.847437 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:59.847402    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:03:59.847930 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:03:59.847912    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:11.930514 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:11.930474    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:04:11.931083 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:11.930752    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" containerID="cri-o://007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970" gracePeriod=30
Jul 16 22:04:11.931083 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:11.930780    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" containerID="cri-o://ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6" gracePeriod=30
Jul 16 22:04:11.931083 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:11.930827    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" containerID="cri-o://f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51" gracePeriod=30
Jul 16 22:04:12.229895 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:12.229814    2551 generic.go:356] "Generic (PLEG): container finished" podID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerID="f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51" exitCode=2
Jul 16 22:04:12.230040 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:12.229893    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerDied","Data":"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"}
Jul 16 22:04:14.842873 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:14.842828    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:17.253575 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:17.253543    2551 generic.go:356] "Generic (PLEG): container finished" podID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerID="007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970" exitCode=0
Jul 16 22:04:17.253958 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:17.253612    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerDied","Data":"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"}
Jul 16 22:04:19.842928 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:19.842879    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:19.847227 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:19.847194    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:04:19.848896 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:19.848871    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:24.843376 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:24.843334    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:24.843762 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:24.843457    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:29.843129 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:29.843089    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:29.847522 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:29.847484    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:04:29.849093 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:29.849063    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:32.051208 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051172    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:04:32.051625 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051583    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent"
Jul 16 22:04:32.051625 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051596    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051628    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="storage-initializer"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051634    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="storage-initializer"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051641    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051647    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051661    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy"
Jul 16 22:04:32.051710 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051667    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy"
Jul 16 22:04:32.051901 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051719    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="agent"
Jul 16 22:04:32.051901 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051728    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kserve-container"
Jul 16 22:04:32.051901 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.051735    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf303239-7ee1-47e5-a191-b778acd68a10" containerName="kube-rbac-proxy"
Jul 16 22:04:32.055338 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.055318    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.057939 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.057916    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 16 22:04:32.058160 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.058127    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 16 22:04:32.066173 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.066104    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:04:32.131021 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.130983    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.131021 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.131019    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.131219 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.131107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.131219 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.131144    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.131219 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.131167    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxld\" (UniqueName: \"kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232446 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232407    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232446 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232448    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232675 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232496    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232675 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232517    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232675 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232535    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxld\" (UniqueName: \"kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.232978 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.232946    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.233325 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.233307    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.233398 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.233331    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.235005 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.234988    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.241234 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.241213    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxld\" (UniqueName: \"kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld\") pod \"isvc-logger-predictor-db9654cd4-qnq9z\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.366271 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.366171    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:32.501967 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:32.501942    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:04:32.504717 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:04:32.504689    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991d7eb8_93b3_42b4_9111_714d0a19ac08.slice/crio-970c4d808a17a05e92d205b4cff3e0b733f06a386e474c729bddde29276c42fa WatchSource:0}: Error finding container 970c4d808a17a05e92d205b4cff3e0b733f06a386e474c729bddde29276c42fa: Status 404 returned error can't find the container with id 970c4d808a17a05e92d205b4cff3e0b733f06a386e474c729bddde29276c42fa
Jul 16 22:04:33.324225 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:33.324185    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerStarted","Data":"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"}
Jul 16 22:04:33.324652 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:33.324277    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerStarted","Data":"970c4d808a17a05e92d205b4cff3e0b733f06a386e474c729bddde29276c42fa"}
Jul 16 22:04:34.843195 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:34.843149    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:37.343340 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:37.343302    2551 generic.go:356] "Generic (PLEG): container finished" podID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerID="b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5" exitCode=0
Jul 16 22:04:37.343720 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:37.343377    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerDied","Data":"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"}
Jul 16 22:04:38.350190 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.350154    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerStarted","Data":"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"}
Jul 16 22:04:38.350587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.350278    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerStarted","Data":"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"}
Jul 16 22:04:38.350587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.350306    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerStarted","Data":"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"}
Jul 16 22:04:38.350666 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.350639    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:38.350788 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.350771    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:38.352118 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.352091    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:04:38.376154 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:38.376109    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podStartSLOduration=2.579735762 podStartE2EDuration="6.376094575s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.796358813s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:04:38.373968516 +0000 UTC m=+861.676141215" watchObservedRunningTime="2026-07-16 22:04:38.376094575 +0000 UTC m=+861.678267248"
Jul 16 22:04:39.354945 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.354911    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:39.355385 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.355006    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:04:39.356171 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.356145    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:39.843530 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.843490    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 16 22:04:39.847810 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.847787    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 16 22:04:39.847901 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.847890    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:39.848533 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.848512    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:39.848624 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:39.848600    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:40.359372 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:40.359328    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:04:40.359964 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:40.359940    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:42.087512 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.087485    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:42.229712 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229619    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location\") pod \"9095d34f-73af-4ff2-8db2-fd05434b67f5\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") "
Jul 16 22:04:42.229881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229714    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config\") pod \"9095d34f-73af-4ff2-8db2-fd05434b67f5\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") "
Jul 16 22:04:42.229881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229738    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls\") pod \"9095d34f-73af-4ff2-8db2-fd05434b67f5\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") "
Jul 16 22:04:42.229881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229757    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert\") pod \"9095d34f-73af-4ff2-8db2-fd05434b67f5\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") "
Jul 16 22:04:42.229881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229795    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp6hm\" (UniqueName: \"kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm\") pod \"9095d34f-73af-4ff2-8db2-fd05434b67f5\" (UID: \"9095d34f-73af-4ff2-8db2-fd05434b67f5\") "
Jul 16 22:04:42.230110 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.229970    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9095d34f-73af-4ff2-8db2-fd05434b67f5" (UID: "9095d34f-73af-4ff2-8db2-fd05434b67f5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:04:42.230110 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.230096    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9095d34f-73af-4ff2-8db2-fd05434b67f5" (UID: "9095d34f-73af-4ff2-8db2-fd05434b67f5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:04:42.230220 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.230124    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9095d34f-73af-4ff2-8db2-fd05434b67f5" (UID: "9095d34f-73af-4ff2-8db2-fd05434b67f5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:04:42.231910 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.231885    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm" (OuterVolumeSpecName: "kube-api-access-tp6hm") pod "9095d34f-73af-4ff2-8db2-fd05434b67f5" (UID: "9095d34f-73af-4ff2-8db2-fd05434b67f5"). InnerVolumeSpecName "kube-api-access-tp6hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:04:42.232019 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.231946    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9095d34f-73af-4ff2-8db2-fd05434b67f5" (UID: "9095d34f-73af-4ff2-8db2-fd05434b67f5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:04:42.330431 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.330398    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9095d34f-73af-4ff2-8db2-fd05434b67f5-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:04:42.330431 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.330424    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:04:42.330661 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.330441    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp6hm\" (UniqueName: \"kubernetes.io/projected/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-api-access-tp6hm\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:04:42.330661 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.330453    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9095d34f-73af-4ff2-8db2-fd05434b67f5-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:04:42.330661 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.330462    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9095d34f-73af-4ff2-8db2-fd05434b67f5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:04:42.370977 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.370941    2551 generic.go:356] "Generic (PLEG): container finished" podID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerID="ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6" exitCode=0
Jul 16 22:04:42.371132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.370980    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerDied","Data":"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"}
Jul 16 22:04:42.371132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.371000    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"
Jul 16 22:04:42.371132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.371047    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9" event={"ID":"9095d34f-73af-4ff2-8db2-fd05434b67f5","Type":"ContainerDied","Data":"86038727f549a5c28b19d89c3f5692c1022df5dbee3b10c1e0790b895beb4418"}
Jul 16 22:04:42.371132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.371091    2551 scope.go:117] "RemoveContainer" containerID="ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"
Jul 16 22:04:42.379893 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.379876    2551 scope.go:117] "RemoveContainer" containerID="f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"
Jul 16 22:04:42.387579 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.387562    2551 scope.go:117] "RemoveContainer" containerID="007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"
Jul 16 22:04:42.395196 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.395180    2551 scope.go:117] "RemoveContainer" containerID="1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"
Jul 16 22:04:42.395749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.395728    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:04:42.400179 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.400159    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-b97b8b77d-tplt9"]
Jul 16 22:04:42.402754 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.402741    2551 scope.go:117] "RemoveContainer" containerID="ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"
Jul 16 22:04:42.402979 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:04:42.402959    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6\": container with ID starting with ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6 not found: ID does not exist" containerID="ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"
Jul 16 22:04:42.403032 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.402986    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6"} err="failed to get container status \"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6\": rpc error: code = NotFound desc = could not find container \"ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6\": container with ID starting with ef38ad98522510a6c271be25359b6d09e892b9d98c0d1eacf1e20404a3487ea6 not found: ID does not exist"
Jul 16 22:04:42.403032 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403001    2551 scope.go:117] "RemoveContainer" containerID="f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"
Jul 16 22:04:42.403243 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:04:42.403224    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51\": container with ID starting with f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51 not found: ID does not exist" containerID="f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"
Jul 16 22:04:42.403326 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403276    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51"} err="failed to get container status \"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51\": rpc error: code = NotFound desc = could not find container \"f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51\": container with ID starting with f0db16f9599813542c27953f4bf4aa3478a58bf53936a5870ecbe4d63e774f51 not found: ID does not exist"
Jul 16 22:04:42.403326 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403300    2551 scope.go:117] "RemoveContainer" containerID="007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"
Jul 16 22:04:42.403531 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:04:42.403516    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970\": container with ID starting with 007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970 not found: ID does not exist" containerID="007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"
Jul 16 22:04:42.403579 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403534    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970"} err="failed to get container status \"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970\": rpc error: code = NotFound desc = could not find container \"007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970\": container with ID starting with 007dee284d20a191566a25d6aa057fdbeb5681452bd19112d636bd3b53094970 not found: ID does not exist"
Jul 16 22:04:42.403579 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403548    2551 scope.go:117] "RemoveContainer" containerID="1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"
Jul 16 22:04:42.403759 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:04:42.403741    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4\": container with ID starting with 1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4 not found: ID does not exist" containerID="1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"
Jul 16 22:04:42.403799 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:42.403765    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4"} err="failed to get container status \"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4\": rpc error: code = NotFound desc = could not find container \"1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4\": container with ID starting with 1d2b9638b61d4322a7747ace864795d72bfb32a6c12d05c6c1fbf597662578e4 not found: ID does not exist"
Jul 16 22:04:43.295587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:43.295557    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" path="/var/lib/kubelet/pods/9095d34f-73af-4ff2-8db2-fd05434b67f5/volumes"
Jul 16 22:04:45.364216 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:45.364184    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:04:45.364876 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:45.364844    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:04:45.365346 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:45.365324    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:04:55.364986 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:55.364938    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:04:55.365537 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:04:55.365465    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:05.365319 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:05.365276    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:05:05.365774 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:05.365750    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:15.365060 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:15.364971    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:05:15.365567 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:15.365435    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:17.180798 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:17.180760    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:05:17.183527 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:17.183505    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:05:25.365513 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:25.365458    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:05:25.365940 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:25.365873    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:35.365147 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:35.365099    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:05:35.365581 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:35.365506    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:45.364816 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:45.364768    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:05:45.365297 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:45.365273    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:05:55.365959 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:55.365927    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:05:55.366523 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:05:55.366134    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:07.426620 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.426586    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:06:07.429235 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.426882    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" containerID="cri-o://64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911" gracePeriod=30
Jul 16 22:06:07.429235 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.426878    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" containerID="cri-o://d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565" gracePeriod=30
Jul 16 22:06:07.429235 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.426914    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" containerID="cri-o://47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57" gracePeriod=30
Jul 16 22:06:07.551392 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551363    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:06:07.551914 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551899    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent"
Jul 16 22:06:07.551964 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551918    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent"
Jul 16 22:06:07.551964 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551943    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="storage-initializer"
Jul 16 22:06:07.551964 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551952    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="storage-initializer"
Jul 16 22:06:07.552062 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551963    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy"
Jul 16 22:06:07.552062 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551973    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy"
Jul 16 22:06:07.552062 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551991    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container"
Jul 16 22:06:07.552062 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.551999    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container"
Jul 16 22:06:07.552226 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.552081    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kserve-container"
Jul 16 22:06:07.552226 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.552097    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="agent"
Jul 16 22:06:07.552226 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.552107    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9095d34f-73af-4ff2-8db2-fd05434b67f5" containerName="kube-rbac-proxy"
Jul 16 22:06:07.555681 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.555657    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.558355 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.558318    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 16 22:06:07.558495 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.558406    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 16 22:06:07.570487 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.570459    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:06:07.660475 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.660445    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.660642 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.660483    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.660642 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.660519    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.660642 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.660589    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkcbs\" (UniqueName: \"kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.660642 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.660605    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761076 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.760995    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761076 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761035    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761364 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761092    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkcbs\" (UniqueName: \"kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761364 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761116    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761364 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761208    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761544 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761492    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.761915 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761894    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.762005 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.761976    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.762195 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.762174    2551 generic.go:356] "Generic (PLEG): container finished" podID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerID="47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57" exitCode=2
Jul 16 22:06:07.762297 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.762273    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerDied","Data":"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"}
Jul 16 22:06:07.763721 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.763705    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.770179 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.770154    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkcbs\" (UniqueName: \"kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:07.867898 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:07.867863    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:06:08.003651 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:08.003627    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:06:08.005661 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:06:08.005632    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4720921_4cb8_48a5_955e_c93ef7f0b753.slice/crio-82101282f3576307df3882504c3948c5109217069ab3990c5e20d903e6a1e164 WatchSource:0}: Error finding container 82101282f3576307df3882504c3948c5109217069ab3990c5e20d903e6a1e164: Status 404 returned error can't find the container with id 82101282f3576307df3882504c3948c5109217069ab3990c5e20d903e6a1e164
Jul 16 22:06:08.767780 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:08.767742    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerStarted","Data":"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"}
Jul 16 22:06:08.768414 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:08.767802    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerStarted","Data":"82101282f3576307df3882504c3948c5109217069ab3990c5e20d903e6a1e164"}
Jul 16 22:06:10.360499 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:10.360455    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:12.786929 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:12.786898    2551 generic.go:356] "Generic (PLEG): container finished" podID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerID="d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565" exitCode=0
Jul 16 22:06:12.787390 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:12.786974    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerDied","Data":"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"}
Jul 16 22:06:15.360615 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:15.360570    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:15.364866 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:15.364827    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:06:15.365191 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:15.365166    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:06:20.360548 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:20.360505    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:20.360953 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:20.360639    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:25.360220 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:25.360177    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:25.365667 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:25.365643    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:06:25.366010 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:25.365979    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:06:30.360153 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:30.360107    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:35.360298 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:35.360232    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Jul 16 22:06:35.364895 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:35.364866    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Jul 16 22:06:35.365023 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:35.365000    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:35.365274 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:35.365237    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:06:35.365366 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:35.365351    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:37.580764 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.580740    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:37.726841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.726755    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config\") pod \"991d7eb8-93b3-42b4-9111-714d0a19ac08\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") "
Jul 16 22:06:37.726841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.726798    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls\") pod \"991d7eb8-93b3-42b4-9111-714d0a19ac08\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") "
Jul 16 22:06:37.726841 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.726821    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location\") pod \"991d7eb8-93b3-42b4-9111-714d0a19ac08\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") "
Jul 16 22:06:37.727145 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.726862    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert\") pod \"991d7eb8-93b3-42b4-9111-714d0a19ac08\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") "
Jul 16 22:06:37.727145 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.726904    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkxld\" (UniqueName: \"kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld\") pod \"991d7eb8-93b3-42b4-9111-714d0a19ac08\" (UID: \"991d7eb8-93b3-42b4-9111-714d0a19ac08\") "
Jul 16 22:06:37.727283 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.727194    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "991d7eb8-93b3-42b4-9111-714d0a19ac08" (UID: "991d7eb8-93b3-42b4-9111-714d0a19ac08"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:06:37.727283 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.727226    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "991d7eb8-93b3-42b4-9111-714d0a19ac08" (UID: "991d7eb8-93b3-42b4-9111-714d0a19ac08"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:06:37.727283 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.727236    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "991d7eb8-93b3-42b4-9111-714d0a19ac08" (UID: "991d7eb8-93b3-42b4-9111-714d0a19ac08"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:06:37.729127 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.729094    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld" (OuterVolumeSpecName: "kube-api-access-jkxld") pod "991d7eb8-93b3-42b4-9111-714d0a19ac08" (UID: "991d7eb8-93b3-42b4-9111-714d0a19ac08"). InnerVolumeSpecName "kube-api-access-jkxld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:06:37.729244 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.729148    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "991d7eb8-93b3-42b4-9111-714d0a19ac08" (UID: "991d7eb8-93b3-42b4-9111-714d0a19ac08"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:06:37.827954 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.827917    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:06:37.827954 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.827950    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkxld\" (UniqueName: \"kubernetes.io/projected/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-api-access-jkxld\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:06:37.828169 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.827965    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/991d7eb8-93b3-42b4-9111-714d0a19ac08-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:06:37.828169 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.827978    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/991d7eb8-93b3-42b4-9111-714d0a19ac08-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:06:37.828169 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.827991    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991d7eb8-93b3-42b4-9111-714d0a19ac08-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:06:37.897681 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.897646    2551 generic.go:356] "Generic (PLEG): container finished" podID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerID="39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d" exitCode=0
Jul 16 22:06:37.897847 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.897717    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerDied","Data":"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"}
Jul 16 22:06:37.899211 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.899195    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:06:37.900245 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.900219    2551 generic.go:356] "Generic (PLEG): container finished" podID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerID="64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911" exitCode=0
Jul 16 22:06:37.900345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.900294    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"
Jul 16 22:06:37.900394 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.900288    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerDied","Data":"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"}
Jul 16 22:06:37.900440 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.900419    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z" event={"ID":"991d7eb8-93b3-42b4-9111-714d0a19ac08","Type":"ContainerDied","Data":"970c4d808a17a05e92d205b4cff3e0b733f06a386e474c729bddde29276c42fa"}
Jul 16 22:06:37.900482 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.900451    2551 scope.go:117] "RemoveContainer" containerID="64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"
Jul 16 22:06:37.910913 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.910899    2551 scope.go:117] "RemoveContainer" containerID="47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"
Jul 16 22:06:37.918431 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.918415    2551 scope.go:117] "RemoveContainer" containerID="d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"
Jul 16 22:06:37.926084 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.926068    2551 scope.go:117] "RemoveContainer" containerID="b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"
Jul 16 22:06:37.933549 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.933533    2551 scope.go:117] "RemoveContainer" containerID="64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"
Jul 16 22:06:37.933849 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:06:37.933827    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911\": container with ID starting with 64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911 not found: ID does not exist" containerID="64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"
Jul 16 22:06:37.933897 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.933858    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911"} err="failed to get container status \"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911\": rpc error: code = NotFound desc = could not find container \"64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911\": container with ID starting with 64c2df1bdadfc35b37418df7db68661e2af929851d6b5678e162ef3399ec2911 not found: ID does not exist"
Jul 16 22:06:37.933897 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.933875    2551 scope.go:117] "RemoveContainer" containerID="47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"
Jul 16 22:06:37.934468 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:06:37.934442    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57\": container with ID starting with 47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57 not found: ID does not exist" containerID="47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"
Jul 16 22:06:37.934622 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.934477    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57"} err="failed to get container status \"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57\": rpc error: code = NotFound desc = could not find container \"47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57\": container with ID starting with 47646ef40fb379a9fd1dc943c86965b40fd1a3a97195fd556f74fd1cb6c43b57 not found: ID does not exist"
Jul 16 22:06:37.934622 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.934501    2551 scope.go:117] "RemoveContainer" containerID="d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"
Jul 16 22:06:37.934909 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:06:37.934851    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565\": container with ID starting with d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565 not found: ID does not exist" containerID="d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"
Jul 16 22:06:37.934909 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.934883    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565"} err="failed to get container status \"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565\": rpc error: code = NotFound desc = could not find container \"d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565\": container with ID starting with d879e451147e827f2e7271da0dfefa816177cf21fd673754a764ac5dcaa6e565 not found: ID does not exist"
Jul 16 22:06:37.934909 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.934908    2551 scope.go:117] "RemoveContainer" containerID="b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"
Jul 16 22:06:37.935202 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:06:37.935178    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5\": container with ID starting with b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5 not found: ID does not exist" containerID="b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"
Jul 16 22:06:37.935274 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.935210    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5"} err="failed to get container status \"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5\": rpc error: code = NotFound desc = could not find container \"b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5\": container with ID starting with b727106b188139cea423d41c7d941493d33a3d3b786bafe6d67056c102ea48c5 not found: ID does not exist"
Jul 16 22:06:37.937106 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.937089    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:06:37.944792 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:37.944770    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-db9654cd4-qnq9z"]
Jul 16 22:06:39.298088 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:06:39.298010    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" path="/var/lib/kubelet/pods/991d7eb8-93b3-42b4-9111-714d0a19ac08/volumes"
Jul 16 22:08:46.532352 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.532314    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerStarted","Data":"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"}
Jul 16 22:08:46.532804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.532385    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerStarted","Data":"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"}
Jul 16 22:08:46.532804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.532420    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:08:46.532804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.532449    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:08:46.533774 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.533747    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 16 22:08:46.564687 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:46.564637    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podStartSLOduration=1.685815545 podStartE2EDuration="2m39.564622982s" totalImagesPullingTime="2m8.199468249s" totalInitContainerRuntime="29.679339188s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:08:46.564354039 +0000 UTC m=+1109.866526734" watchObservedRunningTime="2026-07-16 22:08:46.564622982 +0000 UTC m=+1109.866795654"
Jul 16 22:08:47.536298 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:47.536233    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 16 22:08:52.540628 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:52.540602    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:08:52.541160 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:08:52.541035    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 16 22:09:02.542399 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:02.542368    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:09:07.692362 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:07.692333    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:09:07.692736 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:07.692541    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container" containerID="cri-o://1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293" gracePeriod=30
Jul 16 22:09:07.692736 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:07.692580    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kube-rbac-proxy" containerID="cri-o://edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3" gracePeriod=30
Jul 16 22:09:08.568095 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.568033    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:09:08.603743 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.603713    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkcbs\" (UniqueName: \"kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs\") pod \"d4720921-4cb8-48a5-955e-c93ef7f0b753\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") "
Jul 16 22:09:08.603919 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.603816    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config\") pod \"d4720921-4cb8-48a5-955e-c93ef7f0b753\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") "
Jul 16 22:09:08.603919 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.603851    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location\") pod \"d4720921-4cb8-48a5-955e-c93ef7f0b753\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") "
Jul 16 22:09:08.603919 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.603871    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert\") pod \"d4720921-4cb8-48a5-955e-c93ef7f0b753\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") "
Jul 16 22:09:08.603919 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.603905    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls\") pod \"d4720921-4cb8-48a5-955e-c93ef7f0b753\" (UID: \"d4720921-4cb8-48a5-955e-c93ef7f0b753\") "
Jul 16 22:09:08.604237 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.604212    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d4720921-4cb8-48a5-955e-c93ef7f0b753" (UID: "d4720921-4cb8-48a5-955e-c93ef7f0b753"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:09:08.604319 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.604274    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d4720921-4cb8-48a5-955e-c93ef7f0b753" (UID: "d4720921-4cb8-48a5-955e-c93ef7f0b753"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:09:08.605907 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.605881    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs" (OuterVolumeSpecName: "kube-api-access-zkcbs") pod "d4720921-4cb8-48a5-955e-c93ef7f0b753" (UID: "d4720921-4cb8-48a5-955e-c93ef7f0b753"). InnerVolumeSpecName "kube-api-access-zkcbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:09:08.606089 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.606069    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d4720921-4cb8-48a5-955e-c93ef7f0b753" (UID: "d4720921-4cb8-48a5-955e-c93ef7f0b753"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:09:08.606662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.606636    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d4720921-4cb8-48a5-955e-c93ef7f0b753" (UID: "d4720921-4cb8-48a5-955e-c93ef7f0b753"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:09:08.634258 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634223    2551 generic.go:356] "Generic (PLEG): container finished" podID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerID="edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3" exitCode=2
Jul 16 22:09:08.634387 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634269    2551 generic.go:356] "Generic (PLEG): container finished" podID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerID="1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293" exitCode=0
Jul 16 22:09:08.634387 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634283    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerDied","Data":"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"}
Jul 16 22:09:08.634387 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634308    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"
Jul 16 22:09:08.634387 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634370    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerDied","Data":"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"}
Jul 16 22:09:08.634518 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634394    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2" event={"ID":"d4720921-4cb8-48a5-955e-c93ef7f0b753","Type":"ContainerDied","Data":"82101282f3576307df3882504c3948c5109217069ab3990c5e20d903e6a1e164"}
Jul 16 22:09:08.634518 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.634421    2551 scope.go:117] "RemoveContainer" containerID="edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"
Jul 16 22:09:08.642892 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.642869    2551 scope.go:117] "RemoveContainer" containerID="1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"
Jul 16 22:09:08.650924 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.650902    2551 scope.go:117] "RemoveContainer" containerID="39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"
Jul 16 22:09:08.658144 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.658118    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:09:08.658809 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.658785    2551 scope.go:117] "RemoveContainer" containerID="edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"
Jul 16 22:09:08.659059 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:09:08.659041    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3\": container with ID starting with edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3 not found: ID does not exist" containerID="edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"
Jul 16 22:09:08.659104 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659068    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"} err="failed to get container status \"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3\": rpc error: code = NotFound desc = could not find container \"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3\": container with ID starting with edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3 not found: ID does not exist"
Jul 16 22:09:08.659104 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659085    2551 scope.go:117] "RemoveContainer" containerID="1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"
Jul 16 22:09:08.659345 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:09:08.659328    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293\": container with ID starting with 1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293 not found: ID does not exist" containerID="1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"
Jul 16 22:09:08.659408 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659359    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"} err="failed to get container status \"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293\": rpc error: code = NotFound desc = could not find container \"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293\": container with ID starting with 1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293 not found: ID does not exist"
Jul 16 22:09:08.659408 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659376    2551 scope.go:117] "RemoveContainer" containerID="39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"
Jul 16 22:09:08.659594 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:09:08.659578    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d\": container with ID starting with 39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d not found: ID does not exist" containerID="39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"
Jul 16 22:09:08.659635 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659598    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"} err="failed to get container status \"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d\": rpc error: code = NotFound desc = could not find container \"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d\": container with ID starting with 39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d not found: ID does not exist"
Jul 16 22:09:08.659635 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659612    2551 scope.go:117] "RemoveContainer" containerID="edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"
Jul 16 22:09:08.659817 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659800    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3"} err="failed to get container status \"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3\": rpc error: code = NotFound desc = could not find container \"edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3\": container with ID starting with edd42b30079a3a1c284f85e1cea2aa2d67618a9d55755abd0cead03641c9f0a3 not found: ID does not exist"
Jul 16 22:09:08.659865 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659818    2551 scope.go:117] "RemoveContainer" containerID="1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"
Jul 16 22:09:08.660003 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.659988    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293"} err="failed to get container status \"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293\": rpc error: code = NotFound desc = could not find container \"1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293\": container with ID starting with 1a038230cc2ef79a01d177bb3d3a4cad3bd3dea18819dc0d2445149feb3e2293 not found: ID does not exist"
Jul 16 22:09:08.660050 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.660004    2551 scope.go:117] "RemoveContainer" containerID="39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"
Jul 16 22:09:08.660189 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.660171    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d"} err="failed to get container status \"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d\": rpc error: code = NotFound desc = could not find container \"39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d\": container with ID starting with 39ac116bbd6ca4fadda2ed5792d92a39119fb451ffe70d216ef53e8cf18b045d not found: ID does not exist"
Jul 16 22:09:08.664133 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.664113    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-ngkl2"]
Jul 16 22:09:08.705489 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.705465    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:09:08.705489 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.705488    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4720921-4cb8-48a5-955e-c93ef7f0b753-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:09:08.705854 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.705500    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d4720921-4cb8-48a5-955e-c93ef7f0b753-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:09:08.705854 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.705512    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4720921-4cb8-48a5-955e-c93ef7f0b753-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:09:08.705854 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:08.705520    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkcbs\" (UniqueName: \"kubernetes.io/projected/d4720921-4cb8-48a5-955e-c93ef7f0b753-kube-api-access-zkcbs\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:09:09.294928 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:09.294896    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" path="/var/lib/kubelet/pods/d4720921-4cb8-48a5-955e-c93ef7f0b753/volumes"
Jul 16 22:09:13.038238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038197    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:09:13.038745 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038702    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent"
Jul 16 22:09:13.038745 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038721    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent"
Jul 16 22:09:13.038745 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038737    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container"
Jul 16 22:09:13.038745 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038746    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038771    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038781    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038793    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038803    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038819    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kube-rbac-proxy"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038827    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kube-rbac-proxy"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038842    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="storage-initializer"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038852    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="storage-initializer"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038868    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="storage-initializer"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038876    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="storage-initializer"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038952    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kserve-container"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038964    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4720921-4cb8-48a5-955e-c93ef7f0b753" containerName="kube-rbac-proxy"
Jul 16 22:09:13.038970 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038974    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kserve-container"
Jul 16 22:09:13.039662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038981    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="agent"
Jul 16 22:09:13.039662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.038990    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="991d7eb8-93b3-42b4-9111-714d0a19ac08" containerName="kube-rbac-proxy"
Jul 16 22:09:13.068730 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.068698    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:09:13.068913 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.068837    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.071785 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.071764    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lxc2m\""
Jul 16 22:09:13.072117 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072093    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 16 22:09:13.072230 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072167    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 16 22:09:13.072230 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072181    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 16 22:09:13.072230 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072170    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 16 22:09:13.072990 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072973    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 16 22:09:13.073088 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.072990    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 16 22:09:13.140476 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.140437    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.140668 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.140494    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.140668 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.140593    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25plv\" (UniqueName: \"kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.140668 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.140635    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.140820 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.140685    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.241736 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.241691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.241736 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.241741    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.241969 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.241777    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.241969 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.241812    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.241969 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.241841    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25plv\" (UniqueName: \"kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.242357 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.242329    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.242495 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.242472    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.242561 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.242516    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.244198 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.244180    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.251300 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.251281    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25plv\" (UniqueName: \"kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv\") pod \"isvc-autogluon-v2-predictor-67b849d456-r6hjp\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.384239 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.384157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:13.526968 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.526938    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:09:13.529867 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:09:13.529836    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577f2270_c027_42bb_8fad_4fc40a8842ec.slice/crio-feed748ec37b91c214b944774906ee31484f2ab45fb45b89d725e139b69d1a41 WatchSource:0}: Error finding container feed748ec37b91c214b944774906ee31484f2ab45fb45b89d725e139b69d1a41: Status 404 returned error can't find the container with id feed748ec37b91c214b944774906ee31484f2ab45fb45b89d725e139b69d1a41
Jul 16 22:09:13.658722 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.658631    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerStarted","Data":"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"}
Jul 16 22:09:13.658722 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:13.658697    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerStarted","Data":"feed748ec37b91c214b944774906ee31484f2ab45fb45b89d725e139b69d1a41"}
Jul 16 22:09:33.748146 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:33.748118    2551 generic.go:356] "Generic (PLEG): container finished" podID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerID="437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9" exitCode=0
Jul 16 22:09:33.748525 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:33.748192    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerDied","Data":"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"}
Jul 16 22:09:34.756441 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:34.756406    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerStarted","Data":"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"}
Jul 16 22:09:34.756849 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:34.756470    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerStarted","Data":"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"}
Jul 16 22:09:34.756849 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:34.756654    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:34.780194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:34.780141    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" podStartSLOduration=1.695470075 podStartE2EDuration="21.780124695s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.08465462s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:09:34.777871975 +0000 UTC m=+1158.080044648" watchObservedRunningTime="2026-07-16 22:09:34.780124695 +0000 UTC m=+1158.082297367"
Jul 16 22:09:35.761714 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:35.761686    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:09:41.771880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:09:41.771803    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:10:17.210887 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:10:17.210856    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:10:17.213873 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:10:17.213850    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:11:11.295750 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:11.295679    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:11:13.213492 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:13.213447    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:11:13.214035 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:13.213685    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kserve-container" containerID="cri-o://1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac" gracePeriod=30
Jul 16 22:11:13.214035 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:13.213741    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kube-rbac-proxy" containerID="cri-o://7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536" gracePeriod=30
Jul 16 22:11:14.163124 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.163098    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:11:14.190284 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190232    2551 generic.go:356] "Generic (PLEG): container finished" podID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerID="7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536" exitCode=2
Jul 16 22:11:14.190284 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190282    2551 generic.go:356] "Generic (PLEG): container finished" podID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerID="1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac" exitCode=0
Jul 16 22:11:14.190493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190307    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"
Jul 16 22:11:14.190493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190316    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerDied","Data":"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"}
Jul 16 22:11:14.190493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190405    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerDied","Data":"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"}
Jul 16 22:11:14.190493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190430    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp" event={"ID":"577f2270-c027-42bb-8fad-4fc40a8842ec","Type":"ContainerDied","Data":"feed748ec37b91c214b944774906ee31484f2ab45fb45b89d725e139b69d1a41"}
Jul 16 22:11:14.190493 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.190455    2551 scope.go:117] "RemoveContainer" containerID="7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"
Jul 16 22:11:14.199590 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.199571    2551 scope.go:117] "RemoveContainer" containerID="1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"
Jul 16 22:11:14.208546 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.208528    2551 scope.go:117] "RemoveContainer" containerID="437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"
Jul 16 22:11:14.216095 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.215968    2551 scope.go:117] "RemoveContainer" containerID="7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"
Jul 16 22:11:14.216383 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:11:14.216238    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536\": container with ID starting with 7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536 not found: ID does not exist" containerID="7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"
Jul 16 22:11:14.216383 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216285    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"} err="failed to get container status \"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536\": rpc error: code = NotFound desc = could not find container \"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536\": container with ID starting with 7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536 not found: ID does not exist"
Jul 16 22:11:14.216383 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216306    2551 scope.go:117] "RemoveContainer" containerID="1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"
Jul 16 22:11:14.216538 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:11:14.216522    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac\": container with ID starting with 1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac not found: ID does not exist" containerID="1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"
Jul 16 22:11:14.216582 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216544    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"} err="failed to get container status \"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac\": rpc error: code = NotFound desc = could not find container \"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac\": container with ID starting with 1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac not found: ID does not exist"
Jul 16 22:11:14.216582 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216559    2551 scope.go:117] "RemoveContainer" containerID="437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"
Jul 16 22:11:14.216766 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:11:14.216747    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9\": container with ID starting with 437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9 not found: ID does not exist" containerID="437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"
Jul 16 22:11:14.216819 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216768    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"} err="failed to get container status \"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9\": rpc error: code = NotFound desc = could not find container \"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9\": container with ID starting with 437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9 not found: ID does not exist"
Jul 16 22:11:14.216819 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216780    2551 scope.go:117] "RemoveContainer" containerID="7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"
Jul 16 22:11:14.216964 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216947    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536"} err="failed to get container status \"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536\": rpc error: code = NotFound desc = could not find container \"7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536\": container with ID starting with 7d53c607ec95c66dab88cefb54ec5d259cc62a9146f440820e78b75dab054536 not found: ID does not exist"
Jul 16 22:11:14.217008 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.216965    2551 scope.go:117] "RemoveContainer" containerID="1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"
Jul 16 22:11:14.217143 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.217122    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac"} err="failed to get container status \"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac\": rpc error: code = NotFound desc = could not find container \"1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac\": container with ID starting with 1969292ca625b16c5617f046c48b31e0a11090f4986af5eb66022cd888bfd5ac not found: ID does not exist"
Jul 16 22:11:14.217182 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.217144    2551 scope.go:117] "RemoveContainer" containerID="437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"
Jul 16 22:11:14.217357 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.217334    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9"} err="failed to get container status \"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9\": rpc error: code = NotFound desc = could not find container \"437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9\": container with ID starting with 437428b5a099bca69021c50b24be0f619b495ba7f35ed58f587dd101c43ce6b9 not found: ID does not exist"
Jul 16 22:11:14.271889 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.271798    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls\") pod \"577f2270-c027-42bb-8fad-4fc40a8842ec\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") "
Jul 16 22:11:14.272051 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.271891    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config\") pod \"577f2270-c027-42bb-8fad-4fc40a8842ec\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") "
Jul 16 22:11:14.272051 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.271916    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert\") pod \"577f2270-c027-42bb-8fad-4fc40a8842ec\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") "
Jul 16 22:11:14.272051 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.271951    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location\") pod \"577f2270-c027-42bb-8fad-4fc40a8842ec\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") "
Jul 16 22:11:14.272221 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.272083    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25plv\" (UniqueName: \"kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv\") pod \"577f2270-c027-42bb-8fad-4fc40a8842ec\" (UID: \"577f2270-c027-42bb-8fad-4fc40a8842ec\") "
Jul 16 22:11:14.272414 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.272377    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "577f2270-c027-42bb-8fad-4fc40a8842ec" (UID: "577f2270-c027-42bb-8fad-4fc40a8842ec"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:11:14.272414 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.272411    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "577f2270-c027-42bb-8fad-4fc40a8842ec" (UID: "577f2270-c027-42bb-8fad-4fc40a8842ec"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:11:14.274073 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.274053    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "577f2270-c027-42bb-8fad-4fc40a8842ec" (UID: "577f2270-c027-42bb-8fad-4fc40a8842ec"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:11:14.274208 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.274188    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv" (OuterVolumeSpecName: "kube-api-access-25plv") pod "577f2270-c027-42bb-8fad-4fc40a8842ec" (UID: "577f2270-c027-42bb-8fad-4fc40a8842ec"). InnerVolumeSpecName "kube-api-access-25plv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:11:14.276773 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.276750    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "577f2270-c027-42bb-8fad-4fc40a8842ec" (UID: "577f2270-c027-42bb-8fad-4fc40a8842ec"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:11:14.373194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.373153    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:11:14.373194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.373181    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/577f2270-c027-42bb-8fad-4fc40a8842ec-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:11:14.373194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.373191    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/577f2270-c027-42bb-8fad-4fc40a8842ec-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:11:14.373194 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.373201    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25plv\" (UniqueName: \"kubernetes.io/projected/577f2270-c027-42bb-8fad-4fc40a8842ec-kube-api-access-25plv\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:11:14.373571 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.373210    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/577f2270-c027-42bb-8fad-4fc40a8842ec-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:11:14.513156 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.513124    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:11:14.517724 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:14.517699    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-r6hjp"]
Jul 16 22:11:15.294635 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:15.294603    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" path="/var/lib/kubelet/pods/577f2270-c027-42bb-8fad-4fc40a8842ec/volumes"
Jul 16 22:11:18.477967 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.477933    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478286    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kube-rbac-proxy"
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478298    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kube-rbac-proxy"
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478309    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kserve-container"
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478316    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kserve-container"
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478326    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="storage-initializer"
Jul 16 22:11:18.478345 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478331    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="storage-initializer"
Jul 16 22:11:18.478556 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478385    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kube-rbac-proxy"
Jul 16 22:11:18.478556 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.478394    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="577f2270-c027-42bb-8fad-4fc40a8842ec" containerName="kserve-container"
Jul 16 22:11:18.482647 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.482630    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.485391 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.485365    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 16 22:11:18.485552 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.485365    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lxc2m\""
Jul 16 22:11:18.485552 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.485364    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 16 22:11:18.485552 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.485426    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 16 22:11:18.485552 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.485451    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 16 22:11:18.486425 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.486409    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 16 22:11:18.486425 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.486417    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 16 22:11:18.494130 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.494110    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:11:18.606819 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.606791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.606971 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.606852    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjzg6\" (UniqueName: \"kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.606971 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.606904    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.606971 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.606927    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.606971 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.606960    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708348 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708321    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjzg6\" (UniqueName: \"kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708508 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708372    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708508 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708400    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708508 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708435    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708640 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708510    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.708901 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.708875    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.709133 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.709111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.709211 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.709168    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.710722 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.710706    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.717699 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.717676    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjzg6\" (UniqueName: \"kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.793226 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.793170    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:18.921726 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:18.921690    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:11:18.923460 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:11:18.923433    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c23ef1_8626_4bae_bf90_d9aecceeca66.slice/crio-1945c76436308aefcde8b93c243687b6a05f0e799668d878838fc18af421102f WatchSource:0}: Error finding container 1945c76436308aefcde8b93c243687b6a05f0e799668d878838fc18af421102f: Status 404 returned error can't find the container with id 1945c76436308aefcde8b93c243687b6a05f0e799668d878838fc18af421102f
Jul 16 22:11:19.215211 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:19.215175    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerStarted","Data":"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"}
Jul 16 22:11:19.215441 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:19.215237    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerStarted","Data":"1945c76436308aefcde8b93c243687b6a05f0e799668d878838fc18af421102f"}
Jul 16 22:11:49.345873 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:49.345839    2551 generic.go:356] "Generic (PLEG): container finished" podID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerID="4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285" exitCode=0
Jul 16 22:11:49.346381 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:49.345912    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerDied","Data":"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"}
Jul 16 22:11:50.352930 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:50.352892    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerStarted","Data":"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"}
Jul 16 22:11:50.353399 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:50.352958    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerStarted","Data":"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"}
Jul 16 22:11:50.353399 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:50.353158    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:50.376913 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:50.376861    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" podStartSLOduration=2.984864728 podStartE2EDuration="32.376844618s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.39197989s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:11:50.374622319 +0000 UTC m=+1293.676795005" watchObservedRunningTime="2026-07-16 22:11:50.376844618 +0000 UTC m=+1293.679017290"
Jul 16 22:11:51.357073 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:51.357025    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:11:57.365671 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:11:57.365640    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:13:20.294587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:20.294543    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:13:28.645288 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:28.645231    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:13:28.645804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:28.645499    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kserve-container" containerID="cri-o://dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722" gracePeriod=30
Jul 16 22:13:28.645804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:28.645525    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kube-rbac-proxy" containerID="cri-o://316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8" gracePeriod=30
Jul 16 22:13:28.776172 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:28.776140    2551 generic.go:356] "Generic (PLEG): container finished" podID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerID="316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8" exitCode=2
Jul 16 22:13:28.776330 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:28.776209    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerDied","Data":"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"}
Jul 16 22:13:29.507621 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.507593    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:13:29.543571 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.543539    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjzg6\" (UniqueName: \"kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6\") pod \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") "
Jul 16 22:13:29.543705 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.543618    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls\") pod \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") "
Jul 16 22:13:29.543705 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.543685    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert\") pod \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") "
Jul 16 22:13:29.543819 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.543718    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location\") pod \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") "
Jul 16 22:13:29.543819 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.543744    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config\") pod \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\" (UID: \"e4c23ef1-8626-4bae-bf90-d9aecceeca66\") "
Jul 16 22:13:29.544110 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.544079    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e4c23ef1-8626-4bae-bf90-d9aecceeca66" (UID: "e4c23ef1-8626-4bae-bf90-d9aecceeca66"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:13:29.544110 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.544101    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e4c23ef1-8626-4bae-bf90-d9aecceeca66" (UID: "e4c23ef1-8626-4bae-bf90-d9aecceeca66"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:13:29.545754 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.545734    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e4c23ef1-8626-4bae-bf90-d9aecceeca66" (UID: "e4c23ef1-8626-4bae-bf90-d9aecceeca66"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:13:29.545994 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.545977    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6" (OuterVolumeSpecName: "kube-api-access-mjzg6") pod "e4c23ef1-8626-4bae-bf90-d9aecceeca66" (UID: "e4c23ef1-8626-4bae-bf90-d9aecceeca66"). InnerVolumeSpecName "kube-api-access-mjzg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:13:29.548968 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.548939    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e4c23ef1-8626-4bae-bf90-d9aecceeca66" (UID: "e4c23ef1-8626-4bae-bf90-d9aecceeca66"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:13:29.644749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.644654    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjzg6\" (UniqueName: \"kubernetes.io/projected/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-api-access-mjzg6\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:13:29.644749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.644698    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e4c23ef1-8626-4bae-bf90-d9aecceeca66-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:13:29.644749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.644709    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:13:29.644749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.644717    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:13:29.644749 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.644727    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e4c23ef1-8626-4bae-bf90-d9aecceeca66-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:13:29.782426 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.782383    2551 generic.go:356] "Generic (PLEG): container finished" podID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerID="dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722" exitCode=0
Jul 16 22:13:29.782836 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.782445    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"
Jul 16 22:13:29.782836 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.782471    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerDied","Data":"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"}
Jul 16 22:13:29.782836 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.782533    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8" event={"ID":"e4c23ef1-8626-4bae-bf90-d9aecceeca66","Type":"ContainerDied","Data":"1945c76436308aefcde8b93c243687b6a05f0e799668d878838fc18af421102f"}
Jul 16 22:13:29.782836 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.782561    2551 scope.go:117] "RemoveContainer" containerID="316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"
Jul 16 22:13:29.791735 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.791719    2551 scope.go:117] "RemoveContainer" containerID="dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"
Jul 16 22:13:29.799359 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.799343    2551 scope.go:117] "RemoveContainer" containerID="4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"
Jul 16 22:13:29.806458 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.806442    2551 scope.go:117] "RemoveContainer" containerID="316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"
Jul 16 22:13:29.806727 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:13:29.806705    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8\": container with ID starting with 316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8 not found: ID does not exist" containerID="316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"
Jul 16 22:13:29.806811 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.806734    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8"} err="failed to get container status \"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8\": rpc error: code = NotFound desc = could not find container \"316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8\": container with ID starting with 316bad50961d3c2b91809c45d5a3b78a885a9721ebf52c1efe6fed22e15460b8 not found: ID does not exist"
Jul 16 22:13:29.806811 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.806754    2551 scope.go:117] "RemoveContainer" containerID="dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"
Jul 16 22:13:29.807065 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:13:29.807045    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722\": container with ID starting with dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722 not found: ID does not exist" containerID="dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"
Jul 16 22:13:29.807131 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.807076    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722"} err="failed to get container status \"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722\": rpc error: code = NotFound desc = could not find container \"dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722\": container with ID starting with dde53af1d3e4a375f4609cc188a5bacefcb194d7c4c55f8284da366140599722 not found: ID does not exist"
Jul 16 22:13:29.807131 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.807100    2551 scope.go:117] "RemoveContainer" containerID="4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"
Jul 16 22:13:29.807570 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:13:29.807370    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285\": container with ID starting with 4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285 not found: ID does not exist" containerID="4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"
Jul 16 22:13:29.807570 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.807396    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285"} err="failed to get container status \"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285\": rpc error: code = NotFound desc = could not find container \"4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285\": container with ID starting with 4eb97f8e650f86699e680313155574bf14cbb5e2f7c75b9caf0aad208e5ab285 not found: ID does not exist"
Jul 16 22:13:29.809790 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.809762    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:13:29.813826 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:29.813807    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-nfkz8"]
Jul 16 22:13:31.296051 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:31.296017    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" path="/var/lib/kubelet/pods/e4c23ef1-8626-4bae-bf90-d9aecceeca66/volumes"
Jul 16 22:13:33.798654 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.798616    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:13:33.799434 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799409    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="storage-initializer"
Jul 16 22:13:33.799434 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799437    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="storage-initializer"
Jul 16 22:13:33.799623 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799468    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kserve-container"
Jul 16 22:13:33.799623 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799477    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kserve-container"
Jul 16 22:13:33.799623 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799502    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kube-rbac-proxy"
Jul 16 22:13:33.799623 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799512    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kube-rbac-proxy"
Jul 16 22:13:33.799822 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799694    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kube-rbac-proxy"
Jul 16 22:13:33.799822 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.799720    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c23ef1-8626-4bae-bf90-d9aecceeca66" containerName="kserve-container"
Jul 16 22:13:33.806157 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.806131    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.809386 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.809362    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 16 22:13:33.809712 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.809482    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 16 22:13:33.810501 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.810482    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 16 22:13:33.810788 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.810618    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lxc2m\""
Jul 16 22:13:33.810788 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.810516    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 16 22:13:33.810788 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.810561    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 16 22:13:33.810788 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.810513    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 16 22:13:33.813812 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.813790    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:13:33.879587 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.879550    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.879760 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.879613    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.879760 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.879641    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.879760 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.879669    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtm8l\" (UniqueName: \"kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.879760 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.879700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980309 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980242    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980309 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980313    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980533 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980344    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtm8l\" (UniqueName: \"kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980533 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980378    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980533 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:13:33.980391    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-serving-cert: secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 16 22:13:33.980533 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:13:33.980455    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls podName:98a494f3-3a97-464b-93e5-492aa62ae07d nodeName:}" failed. No retries permitted until 2026-07-16 22:13:34.480437498 +0000 UTC m=+1397.782610149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls") pod "isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d") : secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 16 22:13:33.980533 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980472    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.980833 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.980814    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.981045 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.981027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.981081 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.981027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:33.992476 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:33.992456    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtm8l\" (UniqueName: \"kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:34.485429 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:34.485388    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:34.487773 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:34.487743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:34.717731 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:34.717696    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:34.854782 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:34.854755    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:13:34.857188 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:13:34.857159    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a494f3_3a97_464b_93e5_492aa62ae07d.slice/crio-9b5c893a2d34784c64f4e3fb7d57bcb712beb58b1f14180d969a75392c194489 WatchSource:0}: Error finding container 9b5c893a2d34784c64f4e3fb7d57bcb712beb58b1f14180d969a75392c194489: Status 404 returned error can't find the container with id 9b5c893a2d34784c64f4e3fb7d57bcb712beb58b1f14180d969a75392c194489
Jul 16 22:13:35.816881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:35.816846    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerStarted","Data":"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"}
Jul 16 22:13:35.817056 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:35.816900    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerStarted","Data":"9b5c893a2d34784c64f4e3fb7d57bcb712beb58b1f14180d969a75392c194489"}
Jul 16 22:13:40.840641 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:40.840601    2551 generic.go:356] "Generic (PLEG): container finished" podID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerID="800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8" exitCode=0
Jul 16 22:13:40.841059 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:40.840670    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerDied","Data":"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"}
Jul 16 22:13:41.847805 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:41.847772    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerStarted","Data":"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"}
Jul 16 22:13:41.848298 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:41.847832    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerStarted","Data":"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"}
Jul 16 22:13:41.848298 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:41.848031    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:41.871849 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:41.871795    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" podStartSLOduration=3.9326666059999997 podStartE2EDuration="8.871779212s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.939112606s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:13:41.870493758 +0000 UTC m=+1405.172666433" watchObservedRunningTime="2026-07-16 22:13:41.871779212 +0000 UTC m=+1405.173951885"
Jul 16 22:13:42.852665 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:42.852634    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:13:48.865029 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:13:48.864999    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:15:17.246084 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:17.246052    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:15:17.253484 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:17.253455    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:15:19.295427 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:19.295402    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:15:23.956615 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:23.956584    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:15:23.957144 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:23.956830    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kserve-container" containerID="cri-o://2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e" gracePeriod=30
Jul 16 22:15:23.957144 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:23.956870    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kube-rbac-proxy" containerID="cri-o://bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28" gracePeriod=30
Jul 16 22:15:24.307880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.307789    2551 generic.go:356] "Generic (PLEG): container finished" podID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerID="bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28" exitCode=2
Jul 16 22:15:24.308042 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.307876    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerDied","Data":"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"}
Jul 16 22:15:24.799583 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.799562    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:15:24.935316 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935286    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location\") pod \"98a494f3-3a97-464b-93e5-492aa62ae07d\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") "
Jul 16 22:15:24.935509 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935327    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config\") pod \"98a494f3-3a97-464b-93e5-492aa62ae07d\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") "
Jul 16 22:15:24.935509 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935392    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert\") pod \"98a494f3-3a97-464b-93e5-492aa62ae07d\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") "
Jul 16 22:15:24.935509 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935420    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") pod \"98a494f3-3a97-464b-93e5-492aa62ae07d\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") "
Jul 16 22:15:24.935509 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935436    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtm8l\" (UniqueName: \"kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l\") pod \"98a494f3-3a97-464b-93e5-492aa62ae07d\" (UID: \"98a494f3-3a97-464b-93e5-492aa62ae07d\") "
Jul 16 22:15:24.935817 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935790    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "98a494f3-3a97-464b-93e5-492aa62ae07d" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:15:24.935881 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.935796    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "98a494f3-3a97-464b-93e5-492aa62ae07d" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:15:24.937588 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.937554    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "98a494f3-3a97-464b-93e5-492aa62ae07d" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:15:24.937723 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.937648    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l" (OuterVolumeSpecName: "kube-api-access-wtm8l") pod "98a494f3-3a97-464b-93e5-492aa62ae07d" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d"). InnerVolumeSpecName "kube-api-access-wtm8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:15:24.940427 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:24.940400    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "98a494f3-3a97-464b-93e5-492aa62ae07d" (UID: "98a494f3-3a97-464b-93e5-492aa62ae07d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:15:25.036351 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.036315    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:15:25.036351 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.036343    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a494f3-3a97-464b-93e5-492aa62ae07d-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:15:25.036351 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.036353    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtm8l\" (UniqueName: \"kubernetes.io/projected/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-api-access-wtm8l\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:15:25.036785 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.036362    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a494f3-3a97-464b-93e5-492aa62ae07d-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:15:25.036785 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.036373    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a494f3-3a97-464b-93e5-492aa62ae07d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:15:25.313160 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.313133    2551 generic.go:356] "Generic (PLEG): container finished" podID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerID="2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e" exitCode=0
Jul 16 22:15:25.313310 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.313173    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerDied","Data":"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"}
Jul 16 22:15:25.313310 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.313188    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"
Jul 16 22:15:25.313310 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.313228    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh" event={"ID":"98a494f3-3a97-464b-93e5-492aa62ae07d","Type":"ContainerDied","Data":"9b5c893a2d34784c64f4e3fb7d57bcb712beb58b1f14180d969a75392c194489"}
Jul 16 22:15:25.313310 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.313287    2551 scope.go:117] "RemoveContainer" containerID="bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"
Jul 16 22:15:25.321284 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.321267    2551 scope.go:117] "RemoveContainer" containerID="2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"
Jul 16 22:15:25.328767 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.328747    2551 scope.go:117] "RemoveContainer" containerID="800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"
Jul 16 22:15:25.333083 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.333060    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:15:25.336542 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.336516    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j6zlh"]
Jul 16 22:15:25.337098 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337083    2551 scope.go:117] "RemoveContainer" containerID="bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"
Jul 16 22:15:25.337390 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:15:25.337373    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28\": container with ID starting with bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28 not found: ID does not exist" containerID="bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"
Jul 16 22:15:25.337443 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337402    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28"} err="failed to get container status \"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28\": rpc error: code = NotFound desc = could not find container \"bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28\": container with ID starting with bf7dfe667e771a48de08b589783fcf45132256e1927db6b9af35d051e1303e28 not found: ID does not exist"
Jul 16 22:15:25.337443 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337419    2551 scope.go:117] "RemoveContainer" containerID="2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"
Jul 16 22:15:25.337686 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:15:25.337666    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e\": container with ID starting with 2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e not found: ID does not exist" containerID="2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"
Jul 16 22:15:25.337735 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337691    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e"} err="failed to get container status \"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e\": rpc error: code = NotFound desc = could not find container \"2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e\": container with ID starting with 2a8527a9161fafed6d3483a513c6805369d468abe7edec81d5c230825bff820e not found: ID does not exist"
Jul 16 22:15:25.337735 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337707    2551 scope.go:117] "RemoveContainer" containerID="800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"
Jul 16 22:15:25.337928 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:15:25.337911    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8\": container with ID starting with 800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8 not found: ID does not exist" containerID="800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"
Jul 16 22:15:25.337988 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:25.337932    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8"} err="failed to get container status \"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8\": rpc error: code = NotFound desc = could not find container \"800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8\": container with ID starting with 800584499c318d3090e260e7cc0a9fc93350105580025fb37400cef9928b24d8 not found: ID does not exist"
Jul 16 22:15:27.295335 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:27.295303    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" path="/var/lib/kubelet/pods/98a494f3-3a97-464b-93e5-492aa62ae07d/volumes"
Jul 16 22:15:29.247303 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247270    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:15:29.247796 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247778    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="storage-initializer"
Jul 16 22:15:29.247880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247799    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="storage-initializer"
Jul 16 22:15:29.247880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247813    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kube-rbac-proxy"
Jul 16 22:15:29.247880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247822    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kube-rbac-proxy"
Jul 16 22:15:29.247880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247843    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kserve-container"
Jul 16 22:15:29.247880 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247851    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kserve-container"
Jul 16 22:15:29.248133 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247924    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kube-rbac-proxy"
Jul 16 22:15:29.248133 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.247936    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a494f3-3a97-464b-93e5-492aa62ae07d" containerName="kserve-container"
Jul 16 22:15:29.252875 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.252857    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.255857 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.255831    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 16 22:15:29.255987 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.255877    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 16 22:15:29.255987 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.255934    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 16 22:15:29.257027 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.257011    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lxc2m\""
Jul 16 22:15:29.257132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.257052    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 16 22:15:29.257132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.257079    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 16 22:15:29.257132 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.257115    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 16 22:15:29.262328 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.262310    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:15:29.378260 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.378230    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.378439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.378277    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.378439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.378362    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.378439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.378385    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5gl\" (UniqueName: \"kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.378439 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.378415    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.479593 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.479554    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.479772 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.479600    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.479772 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.479716    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.479900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.479819    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.479900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.479847    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5gl\" (UniqueName: \"kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.480183 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.480157    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.480316 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.480294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.480381 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.480362    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.482073 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.482056    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.488374 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.488353    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5gl\" (UniqueName: \"kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.564471 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.564393    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:29.691759 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:29.691736    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:15:29.693534 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:15:29.693493    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fd13dd_9709_409a_b0f9_16009ead7421.slice/crio-18adf3353d926bbe34430bf06ae0f44d1c5840e88e6fe574f77c4b3e257b8467 WatchSource:0}: Error finding container 18adf3353d926bbe34430bf06ae0f44d1c5840e88e6fe574f77c4b3e257b8467: Status 404 returned error can't find the container with id 18adf3353d926bbe34430bf06ae0f44d1c5840e88e6fe574f77c4b3e257b8467
Jul 16 22:15:30.336897 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:30.336868    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerStarted","Data":"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"}
Jul 16 22:15:30.337291 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:30.336923    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerStarted","Data":"18adf3353d926bbe34430bf06ae0f44d1c5840e88e6fe574f77c4b3e257b8467"}
Jul 16 22:15:39.377647 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:39.377557    2551 generic.go:356] "Generic (PLEG): container finished" podID="16fd13dd-9709-409a-b0f9-16009ead7421" containerID="1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92" exitCode=0
Jul 16 22:15:39.378039 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:39.377636    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerDied","Data":"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"}
Jul 16 22:15:40.385284 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:40.385235    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerStarted","Data":"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"}
Jul 16 22:15:40.385726 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:40.385314    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerStarted","Data":"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"}
Jul 16 22:15:40.385726 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:40.385493    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:40.405050 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:40.404997    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podStartSLOduration=2.77622257 podStartE2EDuration="11.404980267s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.628757697s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:40.403933232 +0000 UTC m=+1523.706105934" watchObservedRunningTime="2026-07-16 22:15:40.404980267 +0000 UTC m=+1523.707152940"
Jul 16 22:15:41.390014 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:41.389977    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:41.391501 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:41.391473    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 16 22:15:42.394466 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:42.394420    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 16 22:15:47.399193 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:47.399162    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:47.399736 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:47.399704    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 16 22:15:57.400263 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:57.400219    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:15:59.225659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:59.225619    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:15:59.226583 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:59.226546    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container" containerID="cri-o://bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f" gracePeriod=30
Jul 16 22:15:59.227119 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:59.227065    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kube-rbac-proxy" containerID="cri-o://10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c" gracePeriod=30
Jul 16 22:15:59.471950 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:59.471915    2551 generic.go:356] "Generic (PLEG): container finished" podID="16fd13dd-9709-409a-b0f9-16009ead7421" containerID="10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c" exitCode=2
Jul 16 22:15:59.472116 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:15:59.471996    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerDied","Data":"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"}
Jul 16 22:16:00.077240 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.077217    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:16:00.140973 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.140941    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location\") pod \"16fd13dd-9709-409a-b0f9-16009ead7421\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") "
Jul 16 22:16:00.141152 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.140988    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf5gl\" (UniqueName: \"kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl\") pod \"16fd13dd-9709-409a-b0f9-16009ead7421\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") "
Jul 16 22:16:00.141152 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141014    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls\") pod \"16fd13dd-9709-409a-b0f9-16009ead7421\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") "
Jul 16 22:16:00.141152 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141074    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config\") pod \"16fd13dd-9709-409a-b0f9-16009ead7421\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") "
Jul 16 22:16:00.141152 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141099    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert\") pod \"16fd13dd-9709-409a-b0f9-16009ead7421\" (UID: \"16fd13dd-9709-409a-b0f9-16009ead7421\") "
Jul 16 22:16:00.141539 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141507    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "16fd13dd-9709-409a-b0f9-16009ead7421" (UID: "16fd13dd-9709-409a-b0f9-16009ead7421"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:16:00.141602 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141517    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "16fd13dd-9709-409a-b0f9-16009ead7421" (UID: "16fd13dd-9709-409a-b0f9-16009ead7421"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:16:00.142019 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.141995    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "16fd13dd-9709-409a-b0f9-16009ead7421" (UID: "16fd13dd-9709-409a-b0f9-16009ead7421"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:16:00.143229 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.143199    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "16fd13dd-9709-409a-b0f9-16009ead7421" (UID: "16fd13dd-9709-409a-b0f9-16009ead7421"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:16:00.143229 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.143222    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl" (OuterVolumeSpecName: "kube-api-access-zf5gl") pod "16fd13dd-9709-409a-b0f9-16009ead7421" (UID: "16fd13dd-9709-409a-b0f9-16009ead7421"). InnerVolumeSpecName "kube-api-access-zf5gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:16:00.242022 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.241990    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:00.242022 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.242023    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16fd13dd-9709-409a-b0f9-16009ead7421-cabundle-cert\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:00.242432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.242036    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16fd13dd-9709-409a-b0f9-16009ead7421-kserve-provision-location\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:00.242432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.242048    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf5gl\" (UniqueName: \"kubernetes.io/projected/16fd13dd-9709-409a-b0f9-16009ead7421-kube-api-access-zf5gl\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:00.242432 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.242057    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/16fd13dd-9709-409a-b0f9-16009ead7421-proxy-tls\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:00.478483 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.478444    2551 generic.go:356] "Generic (PLEG): container finished" podID="16fd13dd-9709-409a-b0f9-16009ead7421" containerID="bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f" exitCode=0
Jul 16 22:16:00.478662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.478504    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"
Jul 16 22:16:00.478662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.478525    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerDied","Data":"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"}
Jul 16 22:16:00.478662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.478593    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n" event={"ID":"16fd13dd-9709-409a-b0f9-16009ead7421","Type":"ContainerDied","Data":"18adf3353d926bbe34430bf06ae0f44d1c5840e88e6fe574f77c4b3e257b8467"}
Jul 16 22:16:00.478662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.478633    2551 scope.go:117] "RemoveContainer" containerID="10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"
Jul 16 22:16:00.487662 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.487641    2551 scope.go:117] "RemoveContainer" containerID="bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"
Jul 16 22:16:00.497416 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.497228    2551 scope.go:117] "RemoveContainer" containerID="1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"
Jul 16 22:16:00.500236 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.500217    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:16:00.503514 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.503491    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-gt69n"]
Jul 16 22:16:00.506562 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.506547    2551 scope.go:117] "RemoveContainer" containerID="10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"
Jul 16 22:16:00.506854 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:00.506831    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c\": container with ID starting with 10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c not found: ID does not exist" containerID="10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"
Jul 16 22:16:00.506900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.506863    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c"} err="failed to get container status \"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c\": rpc error: code = NotFound desc = could not find container \"10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c\": container with ID starting with 10ee530d7109f0abdd1a7496f4158c8b36e5400dd232e079bd36c7196d2d757c not found: ID does not exist"
Jul 16 22:16:00.506900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.506884    2551 scope.go:117] "RemoveContainer" containerID="bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"
Jul 16 22:16:00.507103 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:00.507088    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f\": container with ID starting with bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f not found: ID does not exist" containerID="bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"
Jul 16 22:16:00.507149 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.507107    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f"} err="failed to get container status \"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f\": rpc error: code = NotFound desc = could not find container \"bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f\": container with ID starting with bb9cbcda6900f60fc0c2ff24a89640da598b4e4ecdbe55a27d7ade3d486b1b2f not found: ID does not exist"
Jul 16 22:16:00.507149 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.507120    2551 scope.go:117] "RemoveContainer" containerID="1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"
Jul 16 22:16:00.507316 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:00.507298    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92\": container with ID starting with 1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92 not found: ID does not exist" containerID="1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"
Jul 16 22:16:00.507362 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:00.507319    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92"} err="failed to get container status \"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92\": rpc error: code = NotFound desc = could not find container \"1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92\": container with ID starting with 1c8981f2ff50c8d15512c76fa63a5f906549010c64eb041126de78084ccdae92 not found: ID does not exist"
Jul 16 22:16:01.295329 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:01.295298    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" path="/var/lib/kubelet/pods/16fd13dd-9709-409a-b0f9-16009ead7421/volumes"
Jul 16 22:16:05.819181 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819142    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp6gf/must-gather-9lt72"]
Jul 16 22:16:05.819703 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819685    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container"
Jul 16 22:16:05.819784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819707    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container"
Jul 16 22:16:05.819784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819739    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kube-rbac-proxy"
Jul 16 22:16:05.819784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819750    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kube-rbac-proxy"
Jul 16 22:16:05.819784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819765    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="storage-initializer"
Jul 16 22:16:05.819784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819775    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="storage-initializer"
Jul 16 22:16:05.820038 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819876    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kube-rbac-proxy"
Jul 16 22:16:05.820038 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.819891    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fd13dd-9709-409a-b0f9-16009ead7421" containerName="kserve-container"
Jul 16 22:16:05.824453 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.824434    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.831482 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.831459    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp6gf\"/\"kube-root-ca.crt\""
Jul 16 22:16:05.831594 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.831510    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp6gf\"/\"default-dockercfg-fqdpt\""
Jul 16 22:16:05.832543 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.832529    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp6gf\"/\"openshift-service-ca.crt\""
Jul 16 22:16:05.864593 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.864565    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp6gf/must-gather-9lt72"]
Jul 16 22:16:05.887619 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.887591    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.887753 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.887646    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfwgp\" (UniqueName: \"kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.989071 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.989038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.989225 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.989097    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfwgp\" (UniqueName: \"kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.989436 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.989418    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:05.997018 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:05.996989    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfwgp\" (UniqueName: \"kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp\") pod \"must-gather-9lt72\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") " pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:06.146353 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:06.146245    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:06.274543 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:06.274458    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp6gf/must-gather-9lt72"]
Jul 16 22:16:06.277359 ip-10-0-138-41 kubenswrapper[2551]: W0716 22:16:06.277330    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f2b5eb_d4cf_4811_ae3f_e4b815a24a22.slice/crio-751bfe31a0b4670896544dac6a93306ec5df5ccaf41c673f5acf898d0796451f WatchSource:0}: Error finding container 751bfe31a0b4670896544dac6a93306ec5df5ccaf41c673f5acf898d0796451f: Status 404 returned error can't find the container with id 751bfe31a0b4670896544dac6a93306ec5df5ccaf41c673f5acf898d0796451f
Jul 16 22:16:06.279535 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:06.279519    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:16:06.507589 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:06.507554    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp6gf/must-gather-9lt72" event={"ID":"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22","Type":"ContainerStarted","Data":"751bfe31a0b4670896544dac6a93306ec5df5ccaf41c673f5acf898d0796451f"}
Jul 16 22:16:12.328788 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:12.328753    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f2b5eb_d4cf_4811_ae3f_e4b815a24a22.slice/crio-0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 22:16:12.541522 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:12.541416    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp6gf/must-gather-9lt72" event={"ID":"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22","Type":"ContainerStarted","Data":"bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28"}
Jul 16 22:16:12.541522 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:12.541470    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp6gf/must-gather-9lt72" event={"ID":"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22","Type":"ContainerStarted","Data":"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"}
Jul 16 22:16:12.559652 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:12.559604    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp6gf/must-gather-9lt72" podStartSLOduration=1.614801229 podStartE2EDuration="7.559590129s" totalImagesPullingTime="5.9447889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:16:12.558065158 +0000 UTC m=+1555.860237831" watchObservedRunningTime="2026-07-16 22:16:12.559590129 +0000 UTC m=+1555.861762842"
Jul 16 22:16:32.640498 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:32.640446    2551 generic.go:356] "Generic (PLEG): container finished" podID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerID="0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c" exitCode=0
Jul 16 22:16:32.640950 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:32.640517    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp6gf/must-gather-9lt72" event={"ID":"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22","Type":"ContainerDied","Data":"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"}
Jul 16 22:16:32.640950 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:32.640856    2551 scope.go:117] "RemoveContainer" containerID="0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"
Jul 16 22:16:33.557190 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:33.557164    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp6gf_must-gather-9lt72_20f2b5eb-d4cf-4811-ae3f-e4b815a24a22/gather/0.log"
Jul 16 22:16:36.715743 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:36.715716    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l9vjn_2ab4a654-98ee-45e0-9284-62fe8e871a27/global-pull-secret-syncer/0.log"
Jul 16 22:16:36.901154 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:36.901125    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kvhc6_bae704ef-b67d-40d4-ae3f-b5bd4800861a/konnectivity-agent/0.log"
Jul 16 22:16:36.945210 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:36.945183    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-41.ec2.internal_74669e993430a0a3567c9a05c1f57949/haproxy/0.log"
Jul 16 22:16:38.942852 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:38.942815    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vp6gf/must-gather-9lt72"]
Jul 16 22:16:38.943297 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:38.943007    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-vp6gf/must-gather-9lt72" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="copy" containerID="cri-o://bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28" gracePeriod=2
Jul 16 22:16:38.947278 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:38.946573    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-9lt72\" is forbidden: User \"system:node:ip-10-0-138-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp6gf\": no relationship found between node 'ip-10-0-138-41.ec2.internal' and this object" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:38.947890 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:38.947867    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vp6gf/must-gather-9lt72"]
Jul 16 22:16:39.184020 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.183997    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp6gf_must-gather-9lt72_20f2b5eb-d4cf-4811-ae3f-e4b815a24a22/copy/0.log"
Jul 16 22:16:39.186557 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.186538    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:39.188820 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:39.188792    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-9lt72\" is forbidden: User \"system:node:ip-10-0-138-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp6gf\": no relationship found between node 'ip-10-0-138-41.ec2.internal' and this object" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:39.288693 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.288616    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output\") pod \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") "
Jul 16 22:16:39.288693 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.288679    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfwgp\" (UniqueName: \"kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp\") pod \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\" (UID: \"20f2b5eb-d4cf-4811-ae3f-e4b815a24a22\") "
Jul 16 22:16:39.290050 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.290018    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" (UID: "20f2b5eb-d4cf-4811-ae3f-e4b815a24a22"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:16:39.290916 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.290895    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp" (OuterVolumeSpecName: "kube-api-access-bfwgp") pod "20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" (UID: "20f2b5eb-d4cf-4811-ae3f-e4b815a24a22"). InnerVolumeSpecName "kube-api-access-bfwgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:16:39.295508 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.295473    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" path="/var/lib/kubelet/pods/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22/volumes"
Jul 16 22:16:39.390306 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.390237    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfwgp\" (UniqueName: \"kubernetes.io/projected/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-kube-api-access-bfwgp\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:39.390306 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.390300    2551 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20f2b5eb-d4cf-4811-ae3f-e4b815a24a22-must-gather-output\") on node \"ip-10-0-138-41.ec2.internal\" DevicePath \"\""
Jul 16 22:16:39.672306 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.672218    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp6gf_must-gather-9lt72_20f2b5eb-d4cf-4811-ae3f-e4b815a24a22/copy/0.log"
Jul 16 22:16:39.672660 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.672633    2551 generic.go:356] "Generic (PLEG): container finished" podID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerID="bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28" exitCode=143
Jul 16 22:16:39.672784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.672681    2551 scope.go:117] "RemoveContainer" containerID="bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28"
Jul 16 22:16:39.672784 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.672688    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp6gf/must-gather-9lt72"
Jul 16 22:16:39.680465 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.680452    2551 scope.go:117] "RemoveContainer" containerID="0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"
Jul 16 22:16:39.692645 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.692628    2551 scope.go:117] "RemoveContainer" containerID="bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28"
Jul 16 22:16:39.692898 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:39.692879    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28\": container with ID starting with bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28 not found: ID does not exist" containerID="bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28"
Jul 16 22:16:39.692943 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.692906    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28"} err="failed to get container status \"bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28\": rpc error: code = NotFound desc = could not find container \"bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28\": container with ID starting with bfb6e8ee4eeb927692d735b448ebfade39ea5030bda6a8c168ea618506301a28 not found: ID does not exist"
Jul 16 22:16:39.692943 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.692925    2551 scope.go:117] "RemoveContainer" containerID="0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"
Jul 16 22:16:39.693186 ip-10-0-138-41 kubenswrapper[2551]: E0716 22:16:39.693166    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c\": container with ID starting with 0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c not found: ID does not exist" containerID="0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"
Jul 16 22:16:39.693233 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:39.693192    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c"} err="failed to get container status \"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c\": rpc error: code = NotFound desc = could not find container \"0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c\": container with ID starting with 0130625f3758cc1bf6abffc2a12ab5626cee861729134dc0dfd5964e9330ef7c not found: ID does not exist"
Jul 16 22:16:40.978932 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:40.978899    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/alertmanager/0.log"
Jul 16 22:16:41.025107 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.025084    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/config-reloader/0.log"
Jul 16 22:16:41.050627 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.050601    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/kube-rbac-proxy-web/0.log"
Jul 16 22:16:41.075329 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.075306    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/kube-rbac-proxy/0.log"
Jul 16 22:16:41.099669 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.099640    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/kube-rbac-proxy-metric/0.log"
Jul 16 22:16:41.123058 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.123032    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/prom-label-proxy/0.log"
Jul 16 22:16:41.146157 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.146130    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f6956c2b-8332-40b1-aa07-e89e19d84181/init-config-reloader/0.log"
Jul 16 22:16:41.227271 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.227182    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gqs8z_c53ee81a-c8dc-4504-8448-bb268ee55ec5/kube-state-metrics/0.log"
Jul 16 22:16:41.255892 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.255867    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gqs8z_c53ee81a-c8dc-4504-8448-bb268ee55ec5/kube-rbac-proxy-main/0.log"
Jul 16 22:16:41.284980 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.284952    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gqs8z_c53ee81a-c8dc-4504-8448-bb268ee55ec5/kube-rbac-proxy-self/0.log"
Jul 16 22:16:41.379195 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.379170    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5ks9_08310cbb-fe48-494c-96ad-ff0caf3fee73/node-exporter/0.log"
Jul 16 22:16:41.399703 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.399681    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5ks9_08310cbb-fe48-494c-96ad-ff0caf3fee73/kube-rbac-proxy/0.log"
Jul 16 22:16:41.420772 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.420749    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5ks9_08310cbb-fe48-494c-96ad-ff0caf3fee73/init-textfile/0.log"
Jul 16 22:16:41.614673 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.614598    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vbt6x_8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3/kube-rbac-proxy-main/0.log"
Jul 16 22:16:41.640019 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.639992    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vbt6x_8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3/kube-rbac-proxy-self/0.log"
Jul 16 22:16:41.661633 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.661614    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vbt6x_8cd87e0f-4c24-4cbc-921e-bd7c5164bbb3/openshift-state-metrics/0.log"
Jul 16 22:16:41.702399 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.702372    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/prometheus/0.log"
Jul 16 22:16:41.722463 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.722437    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/config-reloader/0.log"
Jul 16 22:16:41.746411 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.746388    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/thanos-sidecar/0.log"
Jul 16 22:16:41.768086 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.768066    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/kube-rbac-proxy-web/0.log"
Jul 16 22:16:41.793058 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.793040    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/kube-rbac-proxy/0.log"
Jul 16 22:16:41.819634 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.819616    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/kube-rbac-proxy-thanos/0.log"
Jul 16 22:16:41.844756 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:41.844719    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7f818707-5e99-4c32-a111-4322be57f513/init-config-reloader/0.log"
Jul 16 22:16:42.039444 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.039417    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/thanos-query/0.log"
Jul 16 22:16:42.065347 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.065158    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/kube-rbac-proxy-web/0.log"
Jul 16 22:16:42.086224 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.086199    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/kube-rbac-proxy/0.log"
Jul 16 22:16:42.110889 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.110862    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/prom-label-proxy/0.log"
Jul 16 22:16:42.136410 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.136388    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/kube-rbac-proxy-rules/0.log"
Jul 16 22:16:42.155804 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:42.155741    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c86559bf-zqm2v_8eb2219d-128e-4d69-aadc-5f837c09a6cc/kube-rbac-proxy-metrics/0.log"
Jul 16 22:16:43.930420 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:43.930391    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/1.log"
Jul 16 22:16:43.934464 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:43.934446    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-5j647_e38ec207-88d3-4fd1-9e4f-ec9826d265b3/console-operator/2.log"
Jul 16 22:16:44.401747 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:44.401717    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-5xpz8_00ddb4c9-ebf5-4bb5-86cc-95c5360f0bb6/download-server/0.log"
Jul 16 22:16:45.218711 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.218679    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"]
Jul 16 22:16:45.219087 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219015    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="copy"
Jul 16 22:16:45.219087 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219025    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="copy"
Jul 16 22:16:45.219087 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219038    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="gather"
Jul 16 22:16:45.219087 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219045    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="gather"
Jul 16 22:16:45.219217 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219093    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="copy"
Jul 16 22:16:45.219217 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.219103    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f2b5eb-d4cf-4811-ae3f-e4b815a24a22" containerName="gather"
Jul 16 22:16:45.224292 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.224272    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.226900 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.226881    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wsh68\"/\"kube-root-ca.crt\""
Jul 16 22:16:45.227894 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.227877    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wsh68\"/\"openshift-service-ca.crt\""
Jul 16 22:16:45.227974 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.227879    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wsh68\"/\"default-dockercfg-2x8kc\""
Jul 16 22:16:45.232114 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.232092    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"]
Jul 16 22:16:45.234946 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.234925    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-podres\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.235040 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.234952    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zzk\" (UniqueName: \"kubernetes.io/projected/9850e840-0583-460a-9b6d-976f0615a3ee-kube-api-access-b4zzk\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.235040 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.234970    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-proc\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.235040 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.234991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-sys\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.235040 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.235005    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-lib-modules\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.335937 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.335898    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-podres\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336141 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336036    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-podres\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336141 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336062    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zzk\" (UniqueName: \"kubernetes.io/projected/9850e840-0583-460a-9b6d-976f0615a3ee-kube-api-access-b4zzk\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336141 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336104    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-proc\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336323 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336156    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-sys\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336323 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336158    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-proc\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336323 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336188    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-lib-modules\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336323 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336210    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-sys\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.336505 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.336378    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9850e840-0583-460a-9b6d-976f0615a3ee-lib-modules\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.345422 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.345402    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zzk\" (UniqueName: \"kubernetes.io/projected/9850e840-0583-460a-9b6d-976f0615a3ee-kube-api-access-b4zzk\") pod \"perf-node-gather-daemonset-sr525\" (UID: \"9850e840-0583-460a-9b6d-976f0615a3ee\") " pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.535623 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.535518    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:45.667495 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.667469    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"]
Jul 16 22:16:45.698961 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.698931    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525" event={"ID":"9850e840-0583-460a-9b6d-976f0615a3ee","Type":"ContainerStarted","Data":"4d017a72f9c0dcd681beb4cbe544893dd3c182d19cb57dd05e5f25aa088c0992"}
Jul 16 22:16:45.889304 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.889170    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-km54j_61b46972-876c-430c-b9de-187f8f9e9c81/dns/0.log"
Jul 16 22:16:45.911200 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.911162    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-km54j_61b46972-876c-430c-b9de-187f8f9e9c81/kube-rbac-proxy/0.log"
Jul 16 22:16:45.999526 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:45.999492    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n9tkf_d9552277-2e3a-421e-a0d1-4a3131769f8c/dns-node-resolver/0.log"
Jul 16 22:16:46.535602 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:46.535546    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nx9cb_b88a8a6b-c3f2-4b14-a7f2-b6e97c083f91/node-ca/0.log"
Jul 16 22:16:46.704553 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:46.704515    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525" event={"ID":"9850e840-0583-460a-9b6d-976f0615a3ee","Type":"ContainerStarted","Data":"37422d4a88ffa25a119db185d9cbdf8add81fe171e86ecd34629fbbe2292cd9f"}
Jul 16 22:16:46.704748 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:46.704670    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:46.723670 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:46.723618    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525" podStartSLOduration=1.723603743 podStartE2EDuration="1.723603743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:16:46.722485358 +0000 UTC m=+1590.024658041" watchObservedRunningTime="2026-07-16 22:16:46.723603743 +0000 UTC m=+1590.025776417"
Jul 16 22:16:47.365010 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:47.364981    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-748d5b8ddb-97qq5_216a31c0-40ba-4d6a-9c4f-fd734394fe44/router/0.log"
Jul 16 22:16:47.740129 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:47.740070    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lxh7b_59cc2fbf-efaf-4b09-870a-13472c7113c9/serve-healthcheck-canary/0.log"
Jul 16 22:16:48.217619 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:48.217590    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdmzl_b8567367-27fd-45a5-b62b-016d87a108e9/kube-rbac-proxy/0.log"
Jul 16 22:16:48.238349 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:48.238320    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdmzl_b8567367-27fd-45a5-b62b-016d87a108e9/exporter/0.log"
Jul 16 22:16:48.259648 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:48.259623    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdmzl_b8567367-27fd-45a5-b62b-016d87a108e9/extractor/0.log"
Jul 16 22:16:50.706296 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.706270    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-gdqt7_87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5/s3-init/0.log"
Jul 16 22:16:50.721971 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.721948    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-gdqt7_87c1b0e7-3cd5-4c9e-8c49-8759c342bfb5/download-hf-model/0.log"
Jul 16 22:16:50.749469 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.749414    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-cggml_239c97bf-50cf-4971-87e8-dff44c323ed6/s3-tls-init-custom/0.log"
Jul 16 22:16:50.765718 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.765693    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-cggml_239c97bf-50cf-4971-87e8-dff44c323ed6/download-hf-model/0.log"
Jul 16 22:16:50.794148 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.794122    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-wd6nx_907f1a93-e828-48f4-ab5c-e2b04c2e4684/s3-tls-init-serving/0.log"
Jul 16 22:16:50.842373 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:50.842338    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-wd6nx_907f1a93-e828-48f4-ab5c-e2b04c2e4684/download-hf-model/0.log"
Jul 16 22:16:52.719238 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:52.719207    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wsh68/perf-node-gather-daemonset-sr525"
Jul 16 22:16:54.883905 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:54.883872    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-9ncq9_6ef76a2b-ea3d-43d1-b039-673fa844b967/migrator/0.log"
Jul 16 22:16:54.910739 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:54.910714    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-9ncq9_6ef76a2b-ea3d-43d1-b039-673fa844b967/graceful-termination/0.log"
Jul 16 22:16:55.290830 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:55.290799    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6gl8q_1ae625e3-3f8b-40e5-aff6-4bb2a46115c9/kube-storage-version-migrator-operator/1.log"
Jul 16 22:16:55.292480 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:55.292457    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6gl8q_1ae625e3-3f8b-40e5-aff6-4bb2a46115c9/kube-storage-version-migrator-operator/0.log"
Jul 16 22:16:56.208873 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.208839    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6ppht_651589de-cd01-4189-891b-7c30cca28409/kube-multus/0.log"
Jul 16 22:16:56.427794 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.427768    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/kube-multus-additional-cni-plugins/0.log"
Jul 16 22:16:56.450974 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.450952    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/egress-router-binary-copy/0.log"
Jul 16 22:16:56.478419 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.478352    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/cni-plugins/0.log"
Jul 16 22:16:56.502450 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.502414    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/bond-cni-plugin/0.log"
Jul 16 22:16:56.523850 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.523821    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/routeoverride-cni/0.log"
Jul 16 22:16:56.545853 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.545827    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/whereabouts-cni-bincopy/0.log"
Jul 16 22:16:56.569134 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.569111    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dr7m8_e027d183-8aec-4901-b982-4b2c88a803a8/whereabouts-cni/0.log"
Jul 16 22:16:56.847309 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.847226    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2m5v5_5351fd34-a13a-4dc7-84a1-f68100b8c43c/network-metrics-daemon/0.log"
Jul 16 22:16:56.890525 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:56.890498    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2m5v5_5351fd34-a13a-4dc7-84a1-f68100b8c43c/kube-rbac-proxy/0.log"
Jul 16 22:16:58.088244 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.088207    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/ovn-controller/0.log"
Jul 16 22:16:58.110036 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.110011    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/ovn-acl-logging/0.log"
Jul 16 22:16:58.136550 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.136518    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/kube-rbac-proxy-node/0.log"
Jul 16 22:16:58.176933 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.176903    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/kube-rbac-proxy-ovn-metrics/0.log"
Jul 16 22:16:58.218887 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.218863    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/northd/0.log"
Jul 16 22:16:58.268415 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.268391    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/nbdb/0.log"
Jul 16 22:16:58.297446 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.297419    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/sbdb/0.log"
Jul 16 22:16:58.351394 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:58.351321    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t77lx_412fd8b9-d77f-467a-9eb8-7b64e753c9c9/ovnkube-controller/0.log"
Jul 16 22:16:59.344163 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:59.344126    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7bc5b6bf85-5zwt4_15b38079-2e8b-4b95-a15b-5db705392552/check-endpoints/0.log"
Jul 16 22:16:59.418993 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:16:59.418966    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-shstt_1e70a2b4-df90-4dfb-9a10-db3c438750dd/network-check-target-container/0.log"
Jul 16 22:17:00.367072 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:17:00.367044    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-487vh_bba301eb-2534-4f99-be73-7a052e54d48a/iptables-alerter/0.log"
Jul 16 22:17:01.117659 ip-10-0-138-41 kubenswrapper[2551]: I0716 22:17:01.117634    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-6zfwt_8b07ccbe-7598-45cd-b083-0a9e8b857aad/tuned/0.log"