Jul 31 14:47:37.172079 ip-10-0-132-204 systemd[1]: Starting Kubernetes Kubelet...
Jul 31 14:47:37.558154 ip-10-0-132-204 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 14:47:37.558154 ip-10-0-132-204 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 31 14:47:37.558154 ip-10-0-132-204 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 14:47:37.558154 ip-10-0-132-204 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 31 14:47:37.558154 ip-10-0-132-204 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 14:47:37.559102 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.559024    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 14:47:37.563326 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563307    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.563326 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563324    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.563326 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563328    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.563326 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563332    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563337    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563341    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563345    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563349    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563353    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563356    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563360    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563363    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563368    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563372    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563376    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563379    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563390    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563394    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563398    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563402    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563407    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563411    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563415    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.563572 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563419    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563423    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563427    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563431    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563434    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563439    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563443    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563447    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563450    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563455    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563459    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563463    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563468    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563472    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563477    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563482    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563487    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563491    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563495    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563500    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.564417 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563504    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563509    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563513    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563517    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563522    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563527    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563531    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563538    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563544    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563549    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563554    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563559    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563564    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563568    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563572    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563576    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563580    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563584    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563589    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563594    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.565239 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563598    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563603    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563607    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563612    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563616    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563620    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563625    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563632    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563639    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563643    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563648    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563652    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563656    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563660    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563664    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563668    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563672    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563677    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563682    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563686    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.566030 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563690    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563694    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563698    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563702    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563707    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563711    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563716    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563720    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563724    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563728    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563733    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563737    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563742    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563746    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563752    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.563758    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.564998    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565011    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565017    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.566621 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565022    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565026    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565031    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565035    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565040    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565044    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565050    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565054    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565058    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565062    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565067    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565071    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565077    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565081    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565086    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565090    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565095    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565099    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565103    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565107    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.567078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565111    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565115    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565119    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565124    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565128    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565132    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565137    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565141    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565145    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565149    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565153    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565157    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565161    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565165    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565169    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565173    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565178    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565182    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565189    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565194    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.567768 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565199    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565204    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565208    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565213    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565217    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565222    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565226    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565230    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565234    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565238    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565241    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565245    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565249    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565254    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565258    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565263    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565267    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565271    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565275    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.568592 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565282    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565288    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565295    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565299    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565304    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565308    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565312    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565316    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565320    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565324    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565328    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565332    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565337    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565341    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565345    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565349    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565353    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565357    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565361    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565366    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.569420 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565370    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565374    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565378    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565382    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565386    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565391    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565395    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565400    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565404    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565408    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565413    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565417    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565422    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565426    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565434    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565439    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.565443    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567929    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567940    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.570144 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567948    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567954    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567958    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567963    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567967    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567972    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567976    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567980    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567984    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567988    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567992    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.567996    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568000    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568004    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568008    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568012    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568016    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568020    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568024    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568029    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.570593 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568033    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568037    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568041    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568045    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568049    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568053    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568057    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568062    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568067    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568071    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568080    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568084    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568090    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568096    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568101    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568105    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568109    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568115    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568119    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.571146 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568123    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568127    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568132    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568136    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568140    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568144    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568148    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568152    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568157    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568161    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568167    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568173    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568178    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568183    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568187    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568191    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568195    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568199    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568203    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.571779 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568207    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568213    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568217    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568221    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568225    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568229    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568233    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568237    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568241    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568246    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568249    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568253    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568257    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568261    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568265    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568269    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568273    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568277    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568280    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568284    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.572321 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568289    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568293    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568297    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568302    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568306    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568310    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568314    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568318    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568321    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568326    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568330    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568334    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568338    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568342    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568346    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568351    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568355    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568359    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.568363    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568453    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568463    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 31 14:47:37.572786 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568472    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568478    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568484    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568490    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568500    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568506    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568511    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568516    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568521    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568527    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568532    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568537    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568541    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568546    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568551    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568557    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568564    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568568    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568574    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568579    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568584    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568589    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568594    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 31 14:47:37.573399 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568599    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568604    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568609    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568613    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568618    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568623    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568628    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568635    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568639    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568644    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568648    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568653    2566 flags.go:64] FLAG: --enable-server="true"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568658    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568665    2566 flags.go:64] FLAG: --event-burst="100"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568670    2566 flags.go:64] FLAG: --event-qps="50"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568674    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568679    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568684    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568689    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568694    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568698    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568703    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568708    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568713    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568718    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 31 14:47:37.573976 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568723    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568727    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568732    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568736    2566 flags.go:64] FLAG: --feature-gates=""
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568742    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568747    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568751    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568758    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568764    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568768    2566 flags.go:64] FLAG: --help="false"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568773    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568778    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568782    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568787    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568792    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568797    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568819    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568825    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568829    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568834    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568838    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568843    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568847    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568852    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 31 14:47:37.574583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568856    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568861    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568866    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568871    2566 flags.go:64] FLAG: --lock-file=""
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568875    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568880    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568884    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568892    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568898    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568902    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568907    2566 flags.go:64] FLAG: --logging-format="text"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568912    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568917    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568922    2566 flags.go:64] FLAG: --manifest-url=""
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568926    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568934    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568939    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568945    2566 flags.go:64] FLAG: --max-pods="110"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568950    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568955    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568960    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568964    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568969    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568973    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568978    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 31 14:47:37.575167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568988    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568994    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.568998    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569003    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569007    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569016    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569020    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569025    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569030    2566 flags.go:64] FLAG: --port="10250"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569035    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569039    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07c9292546bf7bb85"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569044    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569049    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569053    2566 flags.go:64] FLAG: --register-node="true"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569057    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569063    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569068    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569072    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569077    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569083    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569087    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569092    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569098    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569103    2566 flags.go:64] FLAG: --runonce="false"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569107    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 31 14:47:37.575754 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569112    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569117    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569121    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569125    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569130    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569135    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569139    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569144    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569149    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569153    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569158    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569162    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569167    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569171    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569180    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569184    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569188    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569195    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569199    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569204    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569208    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569213    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569218    2566 flags.go:64] FLAG: --v="2"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569224    2566 flags.go:64] FLAG: --version="false"
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569231    2566 flags.go:64] FLAG: --vmodule=""
Jul 31 14:47:37.576480 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569236    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.569242    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569408    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569414    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569420    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569427    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569432    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569437    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569441    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569445    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569450    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569454    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569459    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569463    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569468    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569473    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569477    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569481    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569485    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.577080 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569489    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569493    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569499    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569503    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569507    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569512    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569516    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569520    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569524    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569528    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569533    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569537    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569541    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569546    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569551    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569555    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569559    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569563    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569567    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569571    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.577516 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569575    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569580    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569584    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569588    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569592    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569597    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569601    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569605    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569609    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569613    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569617    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569621    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569625    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569631    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569640    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569644    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569649    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569653    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569657    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.578007 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569662    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569666    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569670    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569674    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569678    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569682    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569687    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569699    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569703    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569708    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569711    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569715    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569720    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569724    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569728    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569732    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569735    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569739    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569743    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569747    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.578453 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569751    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569755    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569760    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569763    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569767    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569771    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569775    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569781    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569785    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569790    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569794    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569798    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569815    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569820    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569824    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569828    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569835    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569841    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569845    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569849    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.579008 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569856    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569860    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.569863    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.570625    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.577322    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.577336    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.578896    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.578905    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578961    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578966    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578969    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578973    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578976    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578979    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578983    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.579477 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578986    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578988    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578991    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578994    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578996    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.578999    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579001    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579004    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579007    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579009    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579012    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579014    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579017    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579020    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579023    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579025    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579028    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579030    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579033    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579035    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.579849 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579038    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579040    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579049    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579052    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579055    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579057    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579059    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579062    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579064    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579067    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579069    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579072    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579075    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579079    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579082    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579085    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579088    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579090    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579093    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.580307 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579095    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579098    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579100    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579103    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579105    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579108    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579110    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579113    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579115    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579118    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579120    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579122    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579125    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579127    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579129    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579132    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579136    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579140    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579144    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579146    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.580752 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579149    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579152    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579154    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579156    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579159    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579161    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579164    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579166    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579169    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579171    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579173    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579176    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579179    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579181    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579183    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579186    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579188    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579191    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579193    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.581275 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579196    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579198    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579200    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579203    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579205    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579208    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579210    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579213    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579216    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579221    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579227    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579230    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579232    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579235    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.579240    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 14:47:37.581717 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579344    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579348    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579351    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579354    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579356    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579358    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579361    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579364    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579366    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579368    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579371    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579373    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579375    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579378    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579380    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579382    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579385    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579387    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579390    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579392    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 14:47:37.582094 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579395    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579397    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579400    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579404    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579407    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579410    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579414    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579416    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579420    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579422    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579425    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579427    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579430    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579433    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579435    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579437    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579440    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579442    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579444    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 14:47:37.582583 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579447    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579449    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579452    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579454    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579456    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579459    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579461    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579464    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579466    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579469    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579471    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579473    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579476    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579478    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579482    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579485    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579488    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579491    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579494    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 14:47:37.583043 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579497    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579499    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579502    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579505    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579507    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579509    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579512    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579514    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579516    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579519    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579521    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579523    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579526    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579528    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579531    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579534    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579537    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579540    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579543    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579545    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 14:47:37.583476 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579548    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579550    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579552    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579555    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579557    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579559    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579562    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579564    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579566    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579569    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579571    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579574    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579576    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579579    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579581    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579583    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579586    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579589    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579591    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579594    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 14:47:37.583959 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:37.579596    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.579601    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.579701    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.582098    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.583301    2566 server.go:1019] "Starting client certificate rotation"
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.583394    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 14:47:37.584415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.583426    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 14:47:37.585852 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.585839    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 14:47:37.589177 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.589162    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 14:47:37.599550 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.599534    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 31 14:47:37.602043 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.602024    2566 log.go:25] "Validated CRI v1 image API"
Jul 31 14:47:37.602840 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.602827    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 31 14:47:37.606846 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.606826    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e7ce8536-c596-4937-8afa-449f381aaadb:/dev/nvme0n1p4 e8b654b0-bbc3-4d18-a818-6cda902453a9:/dev/nvme0n1p3]
Jul 31 14:47:37.606903 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.606845    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 31 14:47:37.612617 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.612520    2566 manager.go:216] Machine: {Timestamp:2026-07-31 14:47:37.610500912 +0000 UTC m=+0.322475432 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203441 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2874d2f54fa6d5f50596fef7d3c0aa SystemUUID:ec2874d2-f54f-a6d5-f505-96fef7d3c0aa BootID:ca2bd29a-b116-419a-a0c7-7fc2467b24bd Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:be:7d:63:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:be:7d:63:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:da:7e:c3:97:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 31 14:47:37.612617 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.612612    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 31 14:47:37.612719 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.612682    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 31 14:47:37.613742 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.613717    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 14:47:37.613910 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.613746    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 31 14:47:37.615220 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.615208    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 31 14:47:37.615282 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.615225    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 31 14:47:37.615282 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.615237    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 14:47:37.615559 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.615539    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 14:47:37.615756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.615743    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 14:47:37.617609 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.617598    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 31 14:47:37.621575 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621562    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 14:47:37.621705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621687    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 31 14:47:37.621760 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621730    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 31 14:47:37.621760 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621745    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 31 14:47:37.621868 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621765    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 31 14:47:37.621868 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621778    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 31 14:47:37.621868 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.621790    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 14:47:37.622188 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.622175    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 14:47:37.622290 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.622199    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 14:47:37.624768 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.624752    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 31 14:47:37.626374 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.626360    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 31 14:47:37.626420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.626390    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 31 14:47:37.627231 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627220    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 31 14:47:37.627268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627236    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 31 14:47:37.627268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627242    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 31 14:47:37.627268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627248    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 31 14:47:37.627268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627259    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 31 14:47:37.627268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627267    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627279    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627285    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627303    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627308    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627317    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 31 14:47:37.627410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.627322    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 31 14:47:37.628010 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.628001    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 31 14:47:37.628010 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.628010    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 31 14:47:37.631208 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.631193    2566 server.go:1268] "Started kubelet"
Jul 31 14:47:37.631282 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.631243    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 31 14:47:37.631342 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.631274    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 31 14:47:37.631384 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.631348    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 31 14:47:37.632035 ip-10-0-132-204 systemd[1]: Started Kubernetes Kubelet.
Jul 31 14:47:37.635165 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.635141    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 31 14:47:37.636233 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.636202    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 31 14:47:37.636359 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.636343    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 31 14:47:37.636547 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.636518    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 31 14:47:37.637040 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.637016    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 31 14:47:37.637593 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.637567    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 31 14:47:37.641409 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.640492    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-204.ec2.internal.18c76735b446ecb3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-204.ec2.internal,UID:ip-10-0-132-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-204.ec2.internal,},FirstTimestamp:2026-07-31 14:47:37.631165619 +0000 UTC m=+0.343140139,LastTimestamp:2026-07-31 14:47:37.631165619 +0000 UTC m=+0.343140139,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-204.ec2.internal,}"
Jul 31 14:47:37.645226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.645207    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 14:47:37.645226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.645224    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 31 14:47:37.645372 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.645285    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:37.645916 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.645897    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 31 14:47:37.645916 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.645918    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 31 14:47:37.646029 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.645955    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 31 14:47:37.646029 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646026    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 31 14:47:37.646125 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646035    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 31 14:47:37.646125 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646120    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 31 14:47:37.646223 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646133    2566 factory.go:55] Registering systemd factory
Jul 31 14:47:37.646223 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646141    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 31 14:47:37.646367 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646356    2566 factory.go:153] Registering CRI-O factory
Jul 31 14:47:37.646409 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646371    2566 factory.go:223] Registration of the crio container factory successfully
Jul 31 14:47:37.646409 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646390    2566 factory.go:103] Registering Raw factory
Jul 31 14:47:37.646409 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646403    2566 manager.go:1196] Started watching for new ooms in manager
Jul 31 14:47:37.646847 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.646836    2566 manager.go:349] Starting recovery of all containers
Jul 31 14:47:37.657290 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.657258    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 31 14:47:37.657370 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.657325    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 31 14:47:37.657510 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.657496    2566 manager.go:354] Recovery completed
Jul 31 14:47:37.662928 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.662916    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.664795 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.664780    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.664871 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.664826    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.664871 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.664841    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.665364 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.665352    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 31 14:47:37.665364 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.665364    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 31 14:47:37.665492 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.665380    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 14:47:37.665492 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.665407    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.665611 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.665592    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrg42"
Jul 31 14:47:37.666079 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666063    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.666134 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666093    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.666134 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666107    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.666580 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666567    2566 policy_none.go:49] "None policy: Start"
Jul 31 14:47:37.666616 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666587    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 31 14:47:37.666616 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.666596    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 31 14:47:37.667373 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.667258    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-204.ec2.internal.18c76735b6480f5f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-204.ec2.internal,UID:ip-10-0-132-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-204.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-204.ec2.internal,},FirstTimestamp:2026-07-31 14:47:37.664794463 +0000 UTC m=+0.376768982,LastTimestamp:2026-07-31 14:47:37.664794463 +0000 UTC m=+0.376768982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-204.ec2.internal,}"
Jul 31 14:47:37.669415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.669400    2566 policy_none.go:47] "Start"
Jul 31 14:47:37.673771 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.673753    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrg42"
Jul 31 14:47:37.677310 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.677249    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-204.ec2.internal.18c76735b648aa88  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-204.ec2.internal,UID:ip-10-0-132-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-204.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-204.ec2.internal,},FirstTimestamp:2026-07-31 14:47:37.664834184 +0000 UTC m=+0.376808708,LastTimestamp:2026-07-31 14:47:37.664834184 +0000 UTC m=+0.376808708,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-204.ec2.internal,}"
Jul 31 14:47:37.698472 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698459    2566 manager.go:337] "Starting Device Plugin manager"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.698485    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698494    2566 server.go:87] "Starting device plugin registration server"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698707    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698721    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698859    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698945    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.698953    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.699371    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 31 14:47:37.715191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.699423    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:37.799796 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.799771    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.800688 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.800671    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.800765 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.800700    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.800765 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.800712    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.800765 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.800743    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.806791 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.806768    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 31 14:47:37.807965 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.807950    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 31 14:47:37.808025 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.807986    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 31 14:47:37.808025 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.808007    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 31 14:47:37.808115 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.808042    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 31 14:47:37.809801 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.809752    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.809801 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.809784    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-204.ec2.internal\": node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:37.811679 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.811661    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:37.847592 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.847575    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:37.909079 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.908920    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"]
Jul 31 14:47:37.909168 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.909145    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.910066 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.910048    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.910124 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.910082    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.910124 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.910092    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.911506 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.911494    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.911610 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.911596    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.911657 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.911622    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.912147 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912134    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.912217 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912158    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.912217 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912170    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.912310 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912139    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.912310 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912250    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.912310 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.912267    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.913361 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.913345    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.913418 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.913373    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 14:47:37.914023 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.914009    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 14:47:37.914096 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.914032    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 14:47:37.914096 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.914042    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeHasSufficientPID"
Jul 31 14:47:37.937363 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.937340    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-204.ec2.internal\" not found" node="ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.941365 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.941351    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-204.ec2.internal\" not found" node="ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.947527 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.947509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.947608 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.947532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.947608 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:37.947549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346e694eb0cadd3e9407624267497090-config\") pod \"kube-apiserver-proxy-ip-10-0-132-204.ec2.internal\" (UID: \"346e694eb0cadd3e9407624267497090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:37.948550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:37.948534    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.047995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.047967    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.048080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.048030    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.048080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.048051    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.048080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.048073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346e694eb0cadd3e9407624267497090-config\") pod \"kube-apiserver-proxy-ip-10-0-132-204.ec2.internal\" (UID: \"346e694eb0cadd3e9407624267497090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.048173 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.048097    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb7203233120d6ab477b4adb5bea9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal\" (UID: \"71fb7203233120d6ab477b4adb5bea9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.048173 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.048107    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346e694eb0cadd3e9407624267497090-config\") pod \"kube-apiserver-proxy-ip-10-0-132-204.ec2.internal\" (UID: \"346e694eb0cadd3e9407624267497090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.049029 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.049015    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.149201 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.149136    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.240403 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.240360    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.245322 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.245102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.249403 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.249388    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.349863 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.349832    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.450244 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.450182    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.528412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.528385    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:38.550516 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.550490    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.583674 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.583650    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 31 14:47:38.584077 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.583784    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 14:47:38.584077 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.583825    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 14:47:38.650641 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:38.650617    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-204.ec2.internal\" not found"
Jul 31 14:47:38.678982 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.678943    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 14:42:38 +0000 UTC" deadline="2028-04-29 11:45:34.552840546 +0000 UTC"
Jul 31 14:47:38.678982 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.678979    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15308h57m55.873865311s"
Jul 31 14:47:38.681818 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.681795    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:38.746607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.746541    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.764770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.764749    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 14:47:38.765916 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.765903    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal"
Jul 31 14:47:38.776283 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.776260    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 14:47:38.937363 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:38.937329    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fb7203233120d6ab477b4adb5bea9b.slice/crio-bd1ecc803d3e51dd9054e48e62708b2237bc68894ee34541a79a4fb9ab5cc516 WatchSource:0}: Error finding container bd1ecc803d3e51dd9054e48e62708b2237bc68894ee34541a79a4fb9ab5cc516: Status 404 returned error can't find the container with id bd1ecc803d3e51dd9054e48e62708b2237bc68894ee34541a79a4fb9ab5cc516
Jul 31 14:47:38.937565 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:38.937549    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346e694eb0cadd3e9407624267497090.slice/crio-cc20cf94d843b4e1d7a564cde8342eb193c720266e4f498f6a915cdf5d9c6ce8 WatchSource:0}: Error finding container cc20cf94d843b4e1d7a564cde8342eb193c720266e4f498f6a915cdf5d9c6ce8: Status 404 returned error can't find the container with id cc20cf94d843b4e1d7a564cde8342eb193c720266e4f498f6a915cdf5d9c6ce8
Jul 31 14:47:38.942240 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.942225    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 14:47:38.988802 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:38.988778    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:39.575848 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.575805    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:39.622428 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.622401    2566 apiserver.go:52] "Watching apiserver"
Jul 31 14:47:39.632354 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.632333    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 31 14:47:39.634438 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.634398    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-ksbpc","kube-system/global-pull-secret-syncer-vdqg9","kube-system/konnectivity-agent-vfpmp","kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal","openshift-multus/network-metrics-daemon-n8x7h","openshift-network-operator/iptables-alerter-lqtlz","openshift-cluster-node-tuning-operator/tuned-66ct9","openshift-dns/node-resolver-krg8d","openshift-image-registry/node-ca-4bhw7","openshift-multus/multus-additional-cni-plugins-h9nkf","openshift-multus/multus-ndp56","openshift-network-diagnostics/network-check-target-7vzl5"]
Jul 31 14:47:39.635927 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.635894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.637498 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.637206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.638486 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.638462    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.638881 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.638859    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wmt65\""
Jul 31 14:47:39.639196 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.639176    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.639456 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.639437    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.639539 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.639463    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:39.639601 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.639535    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:39.639938 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.639919    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 31 14:47:39.640030 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.639919    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.640676 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.640178    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 31 14:47:39.640676 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.640242    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xpgrz\""
Jul 31 14:47:39.640676 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.640442    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.640939 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.640835    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.641291 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.641209    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 31 14:47:39.642236 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.641732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 31 14:47:39.642354 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.642333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.642945 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.642925    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qfl5j\""
Jul 31 14:47:39.643719 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.643669    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.643806 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.643751    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-88g7r\""
Jul 31 14:47:39.645684 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.645657    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.645877 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.645859    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.645962 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.645934    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 31 14:47:39.646022 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.645993    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 31 14:47:39.646477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.646455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.646558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.646537    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:39.647001 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.646980    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cb6ml\""
Jul 31 14:47:39.647344 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.647314    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.648003 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.647419    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.648003 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.647630    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 31 14:47:39.648416 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.648398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.648791 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.648773    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nrwvf\""
Jul 31 14:47:39.650205 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.649485    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.650205 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.649964    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.650972 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.650952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.651731 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.651709    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 31 14:47:39.651844 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.651752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k5ljw\""
Jul 31 14:47:39.651844 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.651813    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 31 14:47:39.653625 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.653599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.654268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.653828    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 31 14:47:39.654268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.653877    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.654418 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.654274    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 31 14:47:39.654418 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.654283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.655015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.654995    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.655185 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.655162    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:39.655440 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655391    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 31 14:47:39.655440 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655420    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7j449\""
Jul 31 14:47:39.655563 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655547    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 31 14:47:39.655674 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655632    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-tuned\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.655674 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.655794 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f634de43-ec15-469d-95e8-932e660ca0ab-hosts-file\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.655794 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-etc-kubernetes\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.655794 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cnibin\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-agent-certs\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655852    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsc69\" (UniqueName: \"kubernetes.io/projected/69954056-b580-451d-b1f2-806a0c9ec9e3-kube-api-access-nsc69\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-os-release\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-bin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-multus\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-kubernetes\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.655996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-conf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-lib-modules\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-device-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-hostroot\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-multus-certs\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656143    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/69954056-b580-451d-b1f2-806a0c9ec9e3-iptables-alerter-script\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656158    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656173    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-sys\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-etc-selinux\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-cni-binary-copy\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656260    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-run\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-registration-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-netns\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-conf-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjd2\" (UniqueName: \"kubernetes.io/projected/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-kube-api-access-grjd2\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-cnibin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.656641 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656433    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-daemon-config\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-konnectivity-ca\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656499    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656504    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69954056-b580-451d-b1f2-806a0c9ec9e3-host-slash\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvggf\" (UniqueName: \"kubernetes.io/projected/974efb26-338c-4a36-9bac-6c7c8b7bdecc-kube-api-access-fvggf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656653    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxbwz\" (UniqueName: \"kubernetes.io/projected/f634de43-ec15-469d-95e8-932e660ca0ab-kube-api-access-sxbwz\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656725    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-k8s-cni-cncf-io\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656780    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysconfig\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656798    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-79fcl\""
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-sys-fs\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f634de43-ec15-469d-95e8-932e660ca0ab-tmp-dir\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-socket-dir-parent\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-modprobe-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.656996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-var-lib-kubelet\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.657593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657038    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-system-cni-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj8bt\" (UniqueName: \"kubernetes.io/projected/5304b752-64c8-4148-9ba4-ff4c6c5633ce-kube-api-access-lj8bt\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657121    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvxxg\" (UniqueName: \"kubernetes.io/projected/4965b589-30e1-4f14-815e-f6e7b8e07501-kube-api-access-mvxxg\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-socket-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-kubelet\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwhvj\" (UniqueName: \"kubernetes.io/projected/3506513f-4c3f-404c-9a31-ff2f07b34287-kube-api-access-rwhvj\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-system-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-os-release\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-systemd\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-host\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657427    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-tmp\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.658636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.657476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.659281 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.658950    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 14:47:39.679663 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.679642    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5zqvf"
Jul 31 14:47:39.688933 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.688902    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5zqvf"
Jul 31 14:47:39.747156 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.747123    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 31 14:47:39.758068 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758076    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-hostroot\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758099    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-multus-certs\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/69954056-b580-451d-b1f2-806a0c9ec9e3-iptables-alerter-script\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758135    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-sys\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.758204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-etc-selinux\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-cni-binary-copy\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-run\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758323    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-ovn\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-node-log\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovn-node-metrics-cert\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-registration-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-netns\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-conf-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjd2\" (UniqueName: \"kubernetes.io/projected/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-kube-api-access-grjd2\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-cnibin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-hostroot\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-daemon-config\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-konnectivity-ca\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.758607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758595    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69954056-b580-451d-b1f2-806a0c9ec9e3-host-slash\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvggf\" (UniqueName: \"kubernetes.io/projected/974efb26-338c-4a36-9bac-6c7c8b7bdecc-kube-api-access-fvggf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758664    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxbwz\" (UniqueName: \"kubernetes.io/projected/f634de43-ec15-469d-95e8-932e660ca0ab-kube-api-access-sxbwz\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-k8s-cni-cncf-io\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysconfig\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-sys-fs\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-var-lib-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758842    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-log-socket\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758869    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-netd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msl45\" (UniqueName: \"kubernetes.io/projected/6a96c46b-2413-49da-8c07-f5ce93e629c0-kube-api-access-msl45\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f634de43-ec15-469d-95e8-932e660ca0ab-tmp-dir\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-kubelet-config\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-socket-dir-parent\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759013    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-modprobe-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759035    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-var-lib-kubelet\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.759483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-system-cni-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759069    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-systemd-units\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-etc-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759127    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-serviceca\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj8bt\" (UniqueName: \"kubernetes.io/projected/5304b752-64c8-4148-9ba4-ff4c6c5633ce-kube-api-access-lj8bt\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759217    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvxxg\" (UniqueName: \"kubernetes.io/projected/4965b589-30e1-4f14-815e-f6e7b8e07501-kube-api-access-mvxxg\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-config\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759295    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-socket-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759321    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-kubelet\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwhvj\" (UniqueName: \"kubernetes.io/projected/3506513f-4c3f-404c-9a31-ff2f07b34287-kube-api-access-rwhvj\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-kubelet\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-systemd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-system-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-os-release\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.760307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-systemd\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-host\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759546    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-tmp\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759572    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-slash\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759650    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-tuned\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f634de43-ec15-469d-95e8-932e660ca0ab-hosts-file\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759754    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-netns\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759780    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-script-lib\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-host\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wrp\" (UniqueName: \"kubernetes.io/projected/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-kube-api-access-j4wrp\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-etc-kubernetes\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cnibin\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759944    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.759972    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-env-overrides\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760004    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-agent-certs\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsc69\" (UniqueName: \"kubernetes.io/projected/69954056-b580-451d-b1f2-806a0c9ec9e3-kube-api-access-nsc69\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-os-release\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-dbus\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-bin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-multus\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-kubernetes\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760195    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-conf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-lib-modules\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760247    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-device-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-bin\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760615    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760866    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-socket-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.760951    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-sys\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-etc-selinux\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761049    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-system-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.761705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761078    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-kubelet\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-os-release\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761170    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-systemd\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-host\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761246    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-cni-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.758231    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-multus-certs\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/69954056-b580-451d-b1f2-806a0c9ec9e3-iptables-alerter-script\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761492    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-run\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-registration-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-netns\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-conf-dir\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-cni-binary-copy\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761828    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-cnibin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.761871    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.761891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69954056-b580-451d-b1f2-806a0c9ec9e3-host-slash\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.761967    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:47:40.26194434 +0000 UTC m=+2.973918856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-run-k8s-cni-cncf-io\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.762531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762167    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysconfig\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-sys-fs\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-konnectivity-ca\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762360    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-var-lib-kubelet\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762379    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-socket-dir-parent\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3506513f-4c3f-404c-9a31-ff2f07b34287-multus-daemon-config\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-modprobe-d\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cnibin\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762504    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f634de43-ec15-469d-95e8-932e660ca0ab-tmp-dir\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762601    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-bin\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762623    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-etc-kubernetes\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3506513f-4c3f-404c-9a31-ff2f07b34287-host-var-lib-cni-multus\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762660    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-kubernetes\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762698    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965b589-30e1-4f14-815e-f6e7b8e07501-device-dir\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762734    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-system-cni-dir\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.763372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.762745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-sysctl-conf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.764115 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.763659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5304b752-64c8-4148-9ba4-ff4c6c5633ce-os-release\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.764115 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.763769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f634de43-ec15-469d-95e8-932e660ca0ab-hosts-file\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.764355 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.764318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.765572 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.765458    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.765572 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.765523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/974efb26-338c-4a36-9bac-6c7c8b7bdecc-lib-modules\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.766063 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.766044    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-etc-tuned\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.766208 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.766101    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/974efb26-338c-4a36-9bac-6c7c8b7bdecc-tmp\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.766306 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.766203    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5304b752-64c8-4148-9ba4-ff4c6c5633ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.767323 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.767298    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:39.767419 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.767330    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:39.767419 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.767344    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:39.767419 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.767404    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:47:40.267387387 +0000 UTC m=+2.979361923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:39.767581 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.767532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfdcf735-c1a2-407b-9ce7-f32a4b0780dd-agent-certs\") pod \"konnectivity-agent-vfpmp\" (UID: \"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd\") " pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.772282 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.772112    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvxxg\" (UniqueName: \"kubernetes.io/projected/4965b589-30e1-4f14-815e-f6e7b8e07501-kube-api-access-mvxxg\") pod \"aws-ebs-csi-driver-node-75nsv\" (UID: \"4965b589-30e1-4f14-815e-f6e7b8e07501\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.772397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.772374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj8bt\" (UniqueName: \"kubernetes.io/projected/5304b752-64c8-4148-9ba4-ff4c6c5633ce-kube-api-access-lj8bt\") pod \"multus-additional-cni-plugins-h9nkf\" (UID: \"5304b752-64c8-4148-9ba4-ff4c6c5633ce\") " pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:39.772616 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.772593    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvggf\" (UniqueName: \"kubernetes.io/projected/974efb26-338c-4a36-9bac-6c7c8b7bdecc-kube-api-access-fvggf\") pod \"tuned-66ct9\" (UID: \"974efb26-338c-4a36-9bac-6c7c8b7bdecc\") " pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.772769 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.772725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxbwz\" (UniqueName: \"kubernetes.io/projected/f634de43-ec15-469d-95e8-932e660ca0ab-kube-api-access-sxbwz\") pod \"node-resolver-krg8d\" (UID: \"f634de43-ec15-469d-95e8-932e660ca0ab\") " pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.773989 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.773964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjd2\" (UniqueName: \"kubernetes.io/projected/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-kube-api-access-grjd2\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:39.774720 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.774678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwhvj\" (UniqueName: \"kubernetes.io/projected/3506513f-4c3f-404c-9a31-ff2f07b34287-kube-api-access-rwhvj\") pod \"multus-ndp56\" (UID: \"3506513f-4c3f-404c-9a31-ff2f07b34287\") " pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.777178 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.777157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsc69\" (UniqueName: \"kubernetes.io/projected/69954056-b580-451d-b1f2-806a0c9ec9e3-kube-api-access-nsc69\") pod \"iptables-alerter-lqtlz\" (UID: \"69954056-b580-451d-b1f2-806a0c9ec9e3\") " pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.813826 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.813771    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal" event={"ID":"71fb7203233120d6ab477b4adb5bea9b","Type":"ContainerStarted","Data":"bd1ecc803d3e51dd9054e48e62708b2237bc68894ee34541a79a4fb9ab5cc516"}
Jul 31 14:47:39.815267 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.815225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal" event={"ID":"346e694eb0cadd3e9407624267497090","Type":"ContainerStarted","Data":"cc20cf94d843b4e1d7a564cde8342eb193c720266e4f498f6a915cdf5d9c6ce8"}
Jul 31 14:47:39.861179 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-slash\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861179 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861143    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-netns\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861179 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-script-lib\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-host\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wrp\" (UniqueName: \"kubernetes.io/projected/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-kube-api-access-j4wrp\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-slash\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-env-overrides\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-dbus\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-bin\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-ovn\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-node-log\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovn-node-metrics-cert\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861458    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-var-lib-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-log-socket\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-netd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861579    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msl45\" (UniqueName: \"kubernetes.io/projected/6a96c46b-2413-49da-8c07-f5ce93e629c0-kube-api-access-msl45\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-host\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861604    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-kubelet-config\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861634    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-systemd-units\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-netns\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-etc-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-etc-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861761    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-env-overrides\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.861792 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861797    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-dbus\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861834    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-bin\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861905    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-ovn\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.861942    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-node-log\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862058    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862105    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-cni-netd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-var-lib-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-log-socket\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-script-lib\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862180    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.862194    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862209    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-serviceca\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862224    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9bc66d4e-9a34-4922-b231-cf1c4793428a-kubelet-config\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-config\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.862371 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:39.862265    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:47:40.362246161 +0000 UTC m=+3.074220689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862413    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-kubelet\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862495    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-systemd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-openvswitch\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-host-kubelet\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862683    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-run-systemd\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862749    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovnkube-config\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862739    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a96c46b-2413-49da-8c07-f5ce93e629c0-systemd-units\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.863015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.862805    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-serviceca\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.864707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.864677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a96c46b-2413-49da-8c07-f5ce93e629c0-ovn-node-metrics-cert\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.871993 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.871949    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wrp\" (UniqueName: \"kubernetes.io/projected/fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27-kube-api-access-j4wrp\") pod \"node-ca-4bhw7\" (UID: \"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27\") " pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:39.876000 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.875956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msl45\" (UniqueName: \"kubernetes.io/projected/6a96c46b-2413-49da-8c07-f5ce93e629c0-kube-api-access-msl45\") pod \"ovnkube-node-ksbpc\" (UID: \"6a96c46b-2413-49da-8c07-f5ce93e629c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:39.951995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.951966    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krg8d"
Jul 31 14:47:39.960582 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.960560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ndp56"
Jul 31 14:47:39.962639 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:39.962606    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf634de43_ec15_469d_95e8_932e660ca0ab.slice/crio-fe4256963506604fa0f92a2d14c041db66160aa893502061528f8956db511f28 WatchSource:0}: Error finding container fe4256963506604fa0f92a2d14c041db66160aa893502061528f8956db511f28: Status 404 returned error can't find the container with id fe4256963506604fa0f92a2d14c041db66160aa893502061528f8956db511f28
Jul 31 14:47:39.969270 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.969075    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:47:39.969388 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:39.969363    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3506513f_4c3f_404c_9a31_ff2f07b34287.slice/crio-99ec14cee0db1a2811541d6d2bbd00546b9f9a3ec18133867726217f15d45a01 WatchSource:0}: Error finding container 99ec14cee0db1a2811541d6d2bbd00546b9f9a3ec18133867726217f15d45a01: Status 404 returned error can't find the container with id 99ec14cee0db1a2811541d6d2bbd00546b9f9a3ec18133867726217f15d45a01
Jul 31 14:47:39.977824 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.977792    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv"
Jul 31 14:47:39.978547 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:39.978485    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfdcf735_c1a2_407b_9ce7_f32a4b0780dd.slice/crio-ee8e8fbf01e830d1800b6d5627596108d5cce913e59b0021981203b2fb43c2fc WatchSource:0}: Error finding container ee8e8fbf01e830d1800b6d5627596108d5cce913e59b0021981203b2fb43c2fc: Status 404 returned error can't find the container with id ee8e8fbf01e830d1800b6d5627596108d5cce913e59b0021981203b2fb43c2fc
Jul 31 14:47:39.986314 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.986150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqtlz"
Jul 31 14:47:39.987348 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:39.987315    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4965b589_30e1_4f14_815e_f6e7b8e07501.slice/crio-19202a84ba7b7da1e132fae9e556fad86c8a053d63879bb84253b95e375dbd9c WatchSource:0}: Error finding container 19202a84ba7b7da1e132fae9e556fad86c8a053d63879bb84253b95e375dbd9c: Status 404 returned error can't find the container with id 19202a84ba7b7da1e132fae9e556fad86c8a053d63879bb84253b95e375dbd9c
Jul 31 14:47:39.994338 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:39.994178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-66ct9"
Jul 31 14:47:39.994626 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:39.994601    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69954056_b580_451d_b1f2_806a0c9ec9e3.slice/crio-f1e69ca3ab890016b73d2c3346e0170ff94068c95a0fd6d447978e6aba501f42 WatchSource:0}: Error finding container f1e69ca3ab890016b73d2c3346e0170ff94068c95a0fd6d447978e6aba501f42: Status 404 returned error can't find the container with id f1e69ca3ab890016b73d2c3346e0170ff94068c95a0fd6d447978e6aba501f42
Jul 31 14:47:40.002469 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.002442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9nkf"
Jul 31 14:47:40.002990 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:40.002968    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974efb26_338c_4a36_9bac_6c7c8b7bdecc.slice/crio-845212a9912b3cf287ba3999669d3724720c52095f57835f3a0f0afdbc660b64 WatchSource:0}: Error finding container 845212a9912b3cf287ba3999669d3724720c52095f57835f3a0f0afdbc660b64: Status 404 returned error can't find the container with id 845212a9912b3cf287ba3999669d3724720c52095f57835f3a0f0afdbc660b64
Jul 31 14:47:40.010165 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.010146    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:47:40.012376 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:40.012353    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5304b752_64c8_4148_9ba4_ff4c6c5633ce.slice/crio-24675a93402dcd675ce485d04907ea74bcb33ebf2db3d525bb7142fe7f3b8c23 WatchSource:0}: Error finding container 24675a93402dcd675ce485d04907ea74bcb33ebf2db3d525bb7142fe7f3b8c23: Status 404 returned error can't find the container with id 24675a93402dcd675ce485d04907ea74bcb33ebf2db3d525bb7142fe7f3b8c23
Jul 31 14:47:40.017094 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.017074    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bhw7"
Jul 31 14:47:40.020951 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:40.019950    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a96c46b_2413_49da_8c07_f5ce93e629c0.slice/crio-be8d25ab18b2ae6ad1928c24a2359e5207a9df98b856911a31171acf5e6ac1f1 WatchSource:0}: Error finding container be8d25ab18b2ae6ad1928c24a2359e5207a9df98b856911a31171acf5e6ac1f1: Status 404 returned error can't find the container with id be8d25ab18b2ae6ad1928c24a2359e5207a9df98b856911a31171acf5e6ac1f1
Jul 31 14:47:40.027171 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:47:40.027129    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7cce3c_f9e2_450a_ac7f_4f3cbd157f27.slice/crio-f95bd365dbbb075470969550e5a93328922bb1967d3e99b784480726dd0f16c3 WatchSource:0}: Error finding container f95bd365dbbb075470969550e5a93328922bb1967d3e99b784480726dd0f16c3: Status 404 returned error can't find the container with id f95bd365dbbb075470969550e5a93328922bb1967d3e99b784480726dd0f16c3
Jul 31 14:47:40.264965 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.264876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:40.265136 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.265067    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:40.265201 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.265149    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:47:41.26512706 +0000 UTC m=+3.977101573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:40.318004 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.317977    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 14:47:40.365684 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.365645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:40.365886 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.365698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:40.365886 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.365807    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:40.365886 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.365837    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:40.365886 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.365865    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:40.365886 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.365880    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:40.366176 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.365967    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:47:41.36592957 +0000 UTC m=+4.077904085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:40.366176 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.366040    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:47:41.366020536 +0000 UTC m=+4.077995057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:40.692024 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.691813    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 14:42:40 +0000 UTC" deadline="2028-03-19 12:52:56.969313373 +0000 UTC"
Jul 31 14:47:40.692024 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.691870    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14326h5m16.277447245s"
Jul 31 14:47:40.808588 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.808558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:40.808755 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:40.808677    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:40.818220 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.818190    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"be8d25ab18b2ae6ad1928c24a2359e5207a9df98b856911a31171acf5e6ac1f1"}
Jul 31 14:47:40.819490 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.819462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-66ct9" event={"ID":"974efb26-338c-4a36-9bac-6c7c8b7bdecc","Type":"ContainerStarted","Data":"845212a9912b3cf287ba3999669d3724720c52095f57835f3a0f0afdbc660b64"}
Jul 31 14:47:40.820766 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.820738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krg8d" event={"ID":"f634de43-ec15-469d-95e8-932e660ca0ab","Type":"ContainerStarted","Data":"fe4256963506604fa0f92a2d14c041db66160aa893502061528f8956db511f28"}
Jul 31 14:47:40.822064 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.822037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bhw7" event={"ID":"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27","Type":"ContainerStarted","Data":"f95bd365dbbb075470969550e5a93328922bb1967d3e99b784480726dd0f16c3"}
Jul 31 14:47:40.823312 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.823282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerStarted","Data":"24675a93402dcd675ce485d04907ea74bcb33ebf2db3d525bb7142fe7f3b8c23"}
Jul 31 14:47:40.824599 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.824565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqtlz" event={"ID":"69954056-b580-451d-b1f2-806a0c9ec9e3","Type":"ContainerStarted","Data":"f1e69ca3ab890016b73d2c3346e0170ff94068c95a0fd6d447978e6aba501f42"}
Jul 31 14:47:40.825972 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.825930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv" event={"ID":"4965b589-30e1-4f14-815e-f6e7b8e07501","Type":"ContainerStarted","Data":"19202a84ba7b7da1e132fae9e556fad86c8a053d63879bb84253b95e375dbd9c"}
Jul 31 14:47:40.827112 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.827088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vfpmp" event={"ID":"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd","Type":"ContainerStarted","Data":"ee8e8fbf01e830d1800b6d5627596108d5cce913e59b0021981203b2fb43c2fc"}
Jul 31 14:47:40.828355 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:40.828332    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ndp56" event={"ID":"3506513f-4c3f-404c-9a31-ff2f07b34287","Type":"ContainerStarted","Data":"99ec14cee0db1a2811541d6d2bbd00546b9f9a3ec18133867726217f15d45a01"}
Jul 31 14:47:41.273752 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.273725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:41.273946 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.273929    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:41.274027 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.273992    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:47:43.273972678 +0000 UTC m=+5.985947204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:41.374783 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.374756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:41.374862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.374795    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:41.375054 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375015    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:41.375054 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375026    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:41.375054 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375051    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:41.375310 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375065    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:41.375310 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375088    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:47:43.37506698 +0000 UTC m=+6.087041503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:41.375310 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.375117    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:47:43.375100529 +0000 UTC m=+6.087075053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:41.692681 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.692540    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 14:42:40 +0000 UTC" deadline="2028-01-08 11:15:21.821784136 +0000 UTC"
Jul 31 14:47:41.692681 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.692601    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12620h27m40.129187066s"
Jul 31 14:47:41.817015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.816313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:41.817015 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.816434    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:41.817015 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.816847    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:41.817015 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:41.816971    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:41.844434 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.843745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal" event={"ID":"346e694eb0cadd3e9407624267497090","Type":"ContainerStarted","Data":"bd54320f47eb0c951f2cf3a33ae31e7a3a798994c0a55901d8cae96f672bd03d"}
Jul 31 14:47:41.863067 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:41.862679    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-204.ec2.internal" podStartSLOduration=2.862658385 podStartE2EDuration="2.862658385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:47:41.862030738 +0000 UTC m=+4.574005300" watchObservedRunningTime="2026-07-31 14:47:41.862658385 +0000 UTC m=+4.574632918"
Jul 31 14:47:42.808316 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:42.808289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:42.808771 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:42.808414    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:42.859137 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:42.857914    2566 generic.go:356] "Generic (PLEG): container finished" podID="71fb7203233120d6ab477b4adb5bea9b" containerID="a580ce665b44ae7cd11fd5ca0f3eafa8e9d0ac57e321dee27bbb5a356a4ff86a" exitCode=0
Jul 31 14:47:42.859137 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:42.858843    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal" event={"ID":"71fb7203233120d6ab477b4adb5bea9b","Type":"ContainerDied","Data":"a580ce665b44ae7cd11fd5ca0f3eafa8e9d0ac57e321dee27bbb5a356a4ff86a"}
Jul 31 14:47:43.296809 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:43.296771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:43.297014 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.296991    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:43.297136 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.297102    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:47:47.297079879 +0000 UTC m=+10.009054402 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:43.397803 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:43.397765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:43.397992 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:43.397826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:43.397992 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.397982    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:43.398118 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.398044    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:47:47.398025886 +0000 UTC m=+10.110000413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:43.398188 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.398138    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:43.398188 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.398152    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:43.398188 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.398165    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:43.398335 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.398202    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:47:47.398190006 +0000 UTC m=+10.110164559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:43.820900 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:43.820362    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:43.820900 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.820486    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:43.821452 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:43.820944    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:43.821452 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:43.821089    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:44.808401 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:44.808360    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:44.808583 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:44.808502    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:45.812482 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:45.812449    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:45.812976 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:45.812582    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:45.813064 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:45.812993    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:45.813143 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:45.813119    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:46.808945 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:46.808867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:46.809116 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:46.809010    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:47.333877 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:47.333771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:47.334333 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.333923    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:47.334333 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.333993    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:47:55.333971492 +0000 UTC m=+18.045946014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:47.434260 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:47.434224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:47.434289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434399    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434416    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434426    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434461    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:47.434504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434476    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:47:55.434461787 +0000 UTC m=+18.146436300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:47.434836 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.434523    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:47:55.434505737 +0000 UTC m=+18.146480251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:47.810073 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:47.810037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:47.810250 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.810162    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:47.810442 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:47.810389    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:47.810518 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:47.810497    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:48.809172 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:48.809016    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:48.809172 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:48.809156    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:49.809036 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:49.808914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:49.809036 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:49.809034    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:49.809561 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:49.809351    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:49.809561 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:49.809469    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:50.809167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:50.809136    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:50.809342 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:50.809263    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:51.808707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:51.808223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:51.808707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:51.808251    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:51.808707 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:51.808362    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:51.808707 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:51.808501    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:52.809109 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:52.809077    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:52.809587 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:52.809192    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:53.808695 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:53.808621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:53.808695 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:53.808655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:53.808950 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:53.808763    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:53.809121 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:53.809092    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:54.809046 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:54.809014    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:54.809235 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:54.809130    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:55.395032 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:55.394991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:55.395234 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.395124    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:55.395234 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.395201    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:11.395181282 +0000 UTC m=+34.107155798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:47:55.495957 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:55.495920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:55.495957 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:55.495975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:55.496190 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496076    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:55.496190 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496097    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:47:55.496190 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496123    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:47:55.496190 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496136    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:48:11.496118558 +0000 UTC m=+34.208093070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:47:55.496322 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496136    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:55.496322 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.496266    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:11.496252686 +0000 UTC m=+34.208227196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:47:55.808925 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:55.808883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:55.809099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:55.808883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:55.809099 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.809021    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:55.809196 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:55.809115    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:56.809009 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:56.808963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:56.809478 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:56.809109    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:57.809514 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:57.809475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:57.809917 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:57.809586    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:57.809917 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:57.809641    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:57.809917 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:57.809763    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:58.808579 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:58.808546    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:47:58.808766 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:58.808661    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:47:59.808396 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.808242    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:47:59.808842 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.808250    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:47:59.808842 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:59.808528    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:47:59.808842 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:47:59.808574    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:47:59.891760 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.891734    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krg8d" event={"ID":"f634de43-ec15-469d-95e8-932e660ca0ab","Type":"ContainerStarted","Data":"d9c82fe5aaeec7deb760f285643a29b8090031f036403e6758dd4770e1951345"}
Jul 31 14:47:59.893043 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.893023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bhw7" event={"ID":"fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27","Type":"ContainerStarted","Data":"e2ab2c24352c73a8376df5587a64cb8da3b300a82ac5c2c607983c9671f7cff7"}
Jul 31 14:47:59.894339 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.894320    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="778fc81ee96c36572a6af38af4bd6d4af77fd6a6ebdca81a56aabf16d1cfcec2" exitCode=0
Jul 31 14:47:59.894395 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.894384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"778fc81ee96c36572a6af38af4bd6d4af77fd6a6ebdca81a56aabf16d1cfcec2"}
Jul 31 14:47:59.895715 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.895698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv" event={"ID":"4965b589-30e1-4f14-815e-f6e7b8e07501","Type":"ContainerStarted","Data":"c25da288beb3d4f8de12000fed6ef0da0453bbc0336b122338ac173017dd7025"}
Jul 31 14:47:59.897099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.897074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vfpmp" event={"ID":"bfdcf735-c1a2-407b-9ce7-f32a4b0780dd","Type":"ContainerStarted","Data":"7376c45c424486974653ad6576ff8590379d430ce0b045bf9ec73262bd78aede"}
Jul 31 14:47:59.898391 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.898369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ndp56" event={"ID":"3506513f-4c3f-404c-9a31-ff2f07b34287","Type":"ContainerStarted","Data":"b17ab29e6b0530b5081edb04356d3bd8d93bc1d073f4dcfe52331b1eeed08d40"}
Jul 31 14:47:59.900071 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.900047    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal" event={"ID":"71fb7203233120d6ab477b4adb5bea9b","Type":"ContainerStarted","Data":"95bada29c77167c8b93b7210538a683a557fbdb09abb030cd4249624c9d2ce4b"}
Jul 31 14:47:59.902003 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.901982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"4fc8d9ecea4aa8f50c9067170e2118215cd1eadb6baa4172cc13a51c86a761db"}
Jul 31 14:47:59.902085 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.902064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"80e7f7fa3ccf05f7de42f75e41c5a6acc375b2a3515857410023ea931e5a279c"}
Jul 31 14:47:59.903486 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.903460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-66ct9" event={"ID":"974efb26-338c-4a36-9bac-6c7c8b7bdecc","Type":"ContainerStarted","Data":"4f250bfaa69f07c679918c456abebfeb7bd7d3f2309278a2aef8a11bf0ca1b2a"}
Jul 31 14:47:59.940444 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.940248    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-204.ec2.internal" podStartSLOduration=20.940233053 podStartE2EDuration="20.940233053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:47:59.938680451 +0000 UTC m=+22.650654980" watchObservedRunningTime="2026-07-31 14:47:59.940233053 +0000 UTC m=+22.652207581"
Jul 31 14:47:59.956984 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.956939    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4bhw7" podStartSLOduration=1.9079323480000001 podStartE2EDuration="20.956924778s" totalImagesPullingTime="19.04899243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:47:59.956716178 +0000 UTC m=+22.668690706" watchObservedRunningTime="2026-07-31 14:47:59.956924778 +0000 UTC m=+22.668899348"
Jul 31 14:47:59.978125 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.977970    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-krg8d" podStartSLOduration=2.864381287 podStartE2EDuration="21.977955855s" totalImagesPullingTime="19.113574568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:47:59.977908182 +0000 UTC m=+22.689882710" watchObservedRunningTime="2026-07-31 14:47:59.977955855 +0000 UTC m=+22.689930385"
Jul 31 14:47:59.999585 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:47:59.999549    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ndp56" podStartSLOduration=2.771510289 podStartE2EDuration="21.999535225s" totalImagesPullingTime="19.228024936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:47:59.999518764 +0000 UTC m=+22.711493292" watchObservedRunningTime="2026-07-31 14:47:59.999535225 +0000 UTC m=+22.711509753"
Jul 31 14:48:00.020403 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.020369    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vfpmp" podStartSLOduration=2.834032801 podStartE2EDuration="22.020355576s" totalImagesPullingTime="19.186322775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:00.019912468 +0000 UTC m=+22.731886993" watchObservedRunningTime="2026-07-31 14:48:00.020355576 +0000 UTC m=+22.732330103"
Jul 31 14:48:00.809268 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.809234    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:00.809904 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:00.809409    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:00.909707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.909676    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:48:00.910187 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910160    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a96c46b-2413-49da-8c07-f5ce93e629c0" containerID="4fc8d9ecea4aa8f50c9067170e2118215cd1eadb6baa4172cc13a51c86a761db" exitCode=1
Jul 31 14:48:00.910289 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910235    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerDied","Data":"4fc8d9ecea4aa8f50c9067170e2118215cd1eadb6baa4172cc13a51c86a761db"}
Jul 31 14:48:00.910351 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"035b38cd9cdab25327d76e32b9ed537bc9b7be0b0455abc20352ec56ce1d74e1"}
Jul 31 14:48:00.910400 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910366    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"7905480e9f188a5a3086162daea783c0de792a52d7d94e47e03ab8c749f78e23"}
Jul 31 14:48:00.910447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"f6b4f1989d7b9f198011db814c6c5d4b05d917e754d12e9ef7c2eeac2d76ac3a"}
Jul 31 14:48:00.910500 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.910462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"d5c0a1a9a0a431e5be91aeae9db9a7ead975a4127edd589a127e8a50855c2ce2"}
Jul 31 14:48:00.913134 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.912591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqtlz" event={"ID":"69954056-b580-451d-b1f2-806a0c9ec9e3","Type":"ContainerStarted","Data":"03c824888ed857b9c1aa663c59935f7dc6090591190dabb33f247ecb85361eff"}
Jul 31 14:48:00.913134 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.913074    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 31 14:48:00.915031 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.915010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv" event={"ID":"4965b589-30e1-4f14-815e-f6e7b8e07501","Type":"ContainerStarted","Data":"54108ffe108591fd975bdeb430b07dd18f25e7867c45b8212810c49aed3fdbbc"}
Jul 31 14:48:00.937371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:00.937285    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-66ct9" podStartSLOduration=3.7527677280000002 podStartE2EDuration="22.937268617s" totalImagesPullingTime="19.184500889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:00.048562915 +0000 UTC m=+22.760537468" watchObservedRunningTime="2026-07-31 14:48:00.937268617 +0000 UTC m=+23.649243145"
Jul 31 14:48:01.713107 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:01.713001    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T14:48:00.913087624Z","UUID":"547e17f1-4f17-452f-a13e-ed0102695b17","Handler":null,"Name":"","Endpoint":""}
Jul 31 14:48:01.714697 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:01.714677    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 31 14:48:01.714835 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:01.714723    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 31 14:48:01.809086 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:01.809053    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:01.809269 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:01.809104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:01.809269 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:01.809201    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:01.809696 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:01.809312    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:02.809114 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.809083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:02.809287 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:02.809202    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:02.924731 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.921973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv" event={"ID":"4965b589-30e1-4f14-815e-f6e7b8e07501","Type":"ContainerStarted","Data":"2a6a9301d18a575b39fa23f49086d354abb3a86e2b3467ce62ecfc67681843f9"}
Jul 31 14:48:02.928192 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.928170    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:48:02.928557 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.928535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"7c177bbc43d4b74e979ff27bc0d49657e003b05d564c3f127050cab86b6bab7a"}
Jul 31 14:48:02.942583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.942536    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75nsv" podStartSLOduration=2.958195656 podStartE2EDuration="24.942519423s" totalImagesPullingTime="21.984323767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:02.941938893 +0000 UTC m=+25.653913431" watchObservedRunningTime="2026-07-31 14:48:02.942519423 +0000 UTC m=+25.654493952"
Jul 31 14:48:02.942781 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:02.942755    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lqtlz" podStartSLOduration=5.744744898 podStartE2EDuration="24.94274738s" totalImagesPullingTime="19.198002482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:00.937423293 +0000 UTC m=+23.649397822" watchObservedRunningTime="2026-07-31 14:48:02.94274738 +0000 UTC m=+25.654721911"
Jul 31 14:48:03.809261 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:03.809046    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:03.809442 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:03.809115    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:03.809442 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:03.809383    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:03.809568 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:03.809462    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:04.325674 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.325649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:48:04.326217 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.326201    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:48:04.808600 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.808572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:04.808741 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:04.808661    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:04.936627 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.936601    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:48:04.937035 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.937012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"0dd76f7678febc05866d9b69266d6e2d83ce12bdaefcea92313a9b1432b1c422"}
Jul 31 14:48:04.937214 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.937194    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:04.937374 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.937357    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:04.937458 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.937429    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:04.937458 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.937440    2566 scope.go:117] "RemoveContainer" containerID="4fc8d9ecea4aa8f50c9067170e2118215cd1eadb6baa4172cc13a51c86a761db"
Jul 31 14:48:04.939007 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.938985    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="f174aa4d90c88781f9eb32f49369549f3cb52e3c2789051f66fc43461ff8a22d" exitCode=0
Jul 31 14:48:04.939150 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.939068    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"f174aa4d90c88781f9eb32f49369549f3cb52e3c2789051f66fc43461ff8a22d"}
Jul 31 14:48:04.939333 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.939313    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:48:04.940040 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.939927    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vfpmp"
Jul 31 14:48:04.954253 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.954229    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:04.954669 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:04.954656    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:05.808834 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:05.808788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:05.809508 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:05.808909    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:05.809508 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:05.808961    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:05.809508 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:05.809061    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:05.945157 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:05.945131    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:48:05.945594 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:05.945559    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" event={"ID":"6a96c46b-2413-49da-8c07-f5ce93e629c0","Type":"ContainerStarted","Data":"da9b6d4124da3a400ee90aab8d344de5fd490574695dc0efda4eda8bbd71e08e"}
Jul 31 14:48:05.974198 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:05.974153    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc" podStartSLOduration=8.331239583 podStartE2EDuration="27.974136308s" totalImagesPullingTime="19.642896725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:05.972738954 +0000 UTC m=+28.684713510" watchObservedRunningTime="2026-07-31 14:48:05.974136308 +0000 UTC m=+28.686110836"
Jul 31 14:48:06.676617 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.676585    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdqg9"]
Jul 31 14:48:06.676788 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.676701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:06.676867 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:06.676821    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:06.677377 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.677352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n8x7h"]
Jul 31 14:48:06.677464 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.677454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:06.677558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:06.677534    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:06.689720 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.689693    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7vzl5"]
Jul 31 14:48:06.689849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.689785    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:06.689917 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:06.689882    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:06.949435 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.949357    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="0b7767cc25e054ec4bc2eb16eac442e75e76fe50d2fee6698ea1b6ca1bb3e769" exitCode=0
Jul 31 14:48:06.949872 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:06.949442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"0b7767cc25e054ec4bc2eb16eac442e75e76fe50d2fee6698ea1b6ca1bb3e769"}
Jul 31 14:48:08.809163 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:08.809071    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:08.809580 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:08.809169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:08.809580 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:08.809270    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:08.809580 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:08.809172    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:08.809580 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:08.809174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:08.809580 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:08.809390    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:08.955799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:08.955761    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="84d2821f4f1b5e0e58fe84cd341f1f9023cd0dda2e361fa4eb109c5b62de175a" exitCode=0
Jul 31 14:48:08.955980 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:08.955840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"84d2821f4f1b5e0e58fe84cd341f1f9023cd0dda2e361fa4eb109c5b62de175a"}
Jul 31 14:48:10.808620 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:10.808443    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:10.809080 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:10.808730    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdqg9" podUID="9bc66d4e-9a34-4922-b231-cf1c4793428a"
Jul 31 14:48:10.809080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:10.808485    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:10.809080 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:10.808480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:10.809080 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:10.808836    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7vzl5" podUID="4f3ebed1-1af2-4f49-b77e-14efe59b929c"
Jul 31 14:48:10.809080 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:10.808952    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8x7h" podUID="1b29e6e9-f8e0-451e-b658-cbf0ed5ab089"
Jul 31 14:48:11.421099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:11.421069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:11.421389 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.421190    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:48:11.421389 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.421249    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.421235322 +0000 UTC m=+66.133209829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 14:48:11.522368 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:11.522331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:11.522383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522479    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522505    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522528    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret podName:9bc66d4e-9a34-4922-b231-cf1c4793428a nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.522514695 +0000 UTC m=+66.234489205 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret") pod "global-pull-secret-syncer-vdqg9" (UID: "9bc66d4e-9a34-4922-b231-cf1c4793428a") : object "kube-system"/"original-pull-secret" not registered
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522531    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 14:48:11.522550 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522543    2566 projected.go:196] Error preparing data for projected volume kube-api-access-n67vn for pod openshift-network-diagnostics/network-check-target-7vzl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:48:11.522782 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:11.522589    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn podName:4f3ebed1-1af2-4f49-b77e-14efe59b929c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.522574886 +0000 UTC m=+66.234549407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n67vn" (UniqueName: "kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn") pod "network-check-target-7vzl5" (UID: "4f3ebed1-1af2-4f49-b77e-14efe59b929c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 14:48:12.092068 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.091995    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-204.ec2.internal" event="NodeReady"
Jul 31 14:48:12.092496 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.092125    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 31 14:48:12.131789 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.131759    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:48:12.134944 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.134924    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.137516 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.137449    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 31 14:48:12.137703 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.137679    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 31 14:48:12.137798 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.137682    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 31 14:48:12.137798 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.137762    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4fm2\""
Jul 31 14:48:12.145697 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.145676    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 31 14:48:12.152447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.152429    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qq4"]
Jul 31 14:48:12.155853 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.155799    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n2xrq"]
Jul 31 14:48:12.156396 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.156373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.159123 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159104    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 31 14:48:12.159303 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159117    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 31 14:48:12.159824 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:48:12.159909 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159895    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.159979 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159965    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xps48\""
Jul 31 14:48:12.160036 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.159991    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 31 14:48:12.162439 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.162421    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 31 14:48:12.162439 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.162430    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-46zx7\""
Jul 31 14:48:12.162663 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.162464    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 31 14:48:12.168447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.168428    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qq4"]
Jul 31 14:48:12.171839 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.171797    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2xrq"]
Jul 31 14:48:12.229151 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229126    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7lsg\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229286 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229286 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229286 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.229447 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.229368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.316882 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.316838    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log" limit=1024
Jul 31 14:48:12.330594 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330569    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7lsg\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.330725 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.330725 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.330725 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330668    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qrsm\" (UniqueName: \"kubernetes.io/projected/baca90d9-9cb5-43f1-a5b5-47826d395d31-kube-api-access-8qrsm\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.330725 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbd836fa-93a0-4c34-a332-c3606ef36615-cert\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/baca90d9-9cb5-43f1-a5b5-47826d395d31-config-volume\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.330828    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.330846    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.330898    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:12.83088004 +0000 UTC m=+35.542854569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.330968 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xmdv\" (UniqueName: \"kubernetes.io/projected/dbd836fa-93a0-4c34-a332-c3606ef36615-kube-api-access-7xmdv\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.330996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baca90d9-9cb5-43f1-a5b5-47826d395d31-tmp-dir\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.331329 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.331609 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331574    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.331908 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.331883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.335039 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.335020    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.335134 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.335064    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.342328 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.342274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.343666 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.343644    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7lsg\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.409009 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.408979    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log" limit=1024
Jul 31 14:48:12.431578 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xmdv\" (UniqueName: \"kubernetes.io/projected/dbd836fa-93a0-4c34-a332-c3606ef36615-kube-api-access-7xmdv\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.431707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.431707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431604    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baca90d9-9cb5-43f1-a5b5-47826d395d31-tmp-dir\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.431707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qrsm\" (UniqueName: \"kubernetes.io/projected/baca90d9-9cb5-43f1-a5b5-47826d395d31-kube-api-access-8qrsm\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.431874 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.431705    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:12.431874 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbd836fa-93a0-4c34-a332-c3606ef36615-cert\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.431874 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.431743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/baca90d9-9cb5-43f1-a5b5-47826d395d31-config-volume\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.431874 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.431765    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:12.931745421 +0000 UTC m=+35.643719930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:12.432109 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.432055    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baca90d9-9cb5-43f1-a5b5-47826d395d31-tmp-dir\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.432411 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.432391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/baca90d9-9cb5-43f1-a5b5-47826d395d31-config-volume\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.434136 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.434111    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbd836fa-93a0-4c34-a332-c3606ef36615-cert\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.442788 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.442764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qrsm\" (UniqueName: \"kubernetes.io/projected/baca90d9-9cb5-43f1-a5b5-47826d395d31-kube-api-access-8qrsm\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.442894 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.442821    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xmdv\" (UniqueName: \"kubernetes.io/projected/dbd836fa-93a0-4c34-a332-c3606ef36615-kube-api-access-7xmdv\") pod \"ingress-canary-g5qq4\" (UID: \"dbd836fa-93a0-4c34-a332-c3606ef36615\") " pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.466853 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.466834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5qq4"
Jul 31 14:48:12.808475 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.808439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:12.808664 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.808484    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:12.808664 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.808484    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:12.811306 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811286    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4v8n\""
Jul 31 14:48:12.811426 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811327    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 14:48:12.811426 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811364    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2kc8\""
Jul 31 14:48:12.811714 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811585    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 14:48:12.811714 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811612    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 14:48:12.811714 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.811618    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 14:48:12.835895 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.835876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:12.836022 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.836010    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:12.836022 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.836023    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:12.836093 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.836065    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:13.836054043 +0000 UTC m=+36.548028563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:12.936829 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:12.936789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:12.936993 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.936948    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:12.937056 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:12.937018    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:13.936998528 +0000 UTC m=+36.648973036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:13.845186 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:13.845156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:13.845770 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:13.845562    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:13.845770 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:13.845587    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:13.845770 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:13.845673    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:15.84564083 +0000 UTC m=+38.557615350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:13.946710 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:13.946555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:13.946710 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:13.946699    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:13.946902 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:13.946758    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:15.946743289 +0000 UTC m=+38.658717819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:13.956244 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:13.956042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qq4"]
Jul 31 14:48:13.959881 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:13.959855    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd836fa_93a0_4c34_a332_c3606ef36615.slice/crio-bea85b911414fe9e151d0dc7822d39ab2c07fb2bb8135d66ee42e4f7d94d90d1 WatchSource:0}: Error finding container bea85b911414fe9e151d0dc7822d39ab2c07fb2bb8135d66ee42e4f7d94d90d1: Status 404 returned error can't find the container with id bea85b911414fe9e151d0dc7822d39ab2c07fb2bb8135d66ee42e4f7d94d90d1
Jul 31 14:48:13.967899 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:13.967874    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5qq4" event={"ID":"dbd836fa-93a0-4c34-a332-c3606ef36615","Type":"ContainerStarted","Data":"bea85b911414fe9e151d0dc7822d39ab2c07fb2bb8135d66ee42e4f7d94d90d1"}
Jul 31 14:48:14.973367 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:14.973324    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="897a1293e59de748ed6c6fde12aa559609ce3b62a2c56ed245e57367d0f3b431" exitCode=0
Jul 31 14:48:14.973863 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:14.973371    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"897a1293e59de748ed6c6fde12aa559609ce3b62a2c56ed245e57367d0f3b431"}
Jul 31 14:48:15.860388 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:15.860352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:15.860559 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:15.860522    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:15.860559 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:15.860542    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:15.860657 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:15.860608    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:19.860589351 +0000 UTC m=+42.572563879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:15.961411 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:15.961384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:15.961584 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:15.961560    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:15.961668 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:15.961642    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:19.9616218 +0000 UTC m=+42.673596326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:15.980235 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:15.980205    2566 generic.go:356] "Generic (PLEG): container finished" podID="5304b752-64c8-4148-9ba4-ff4c6c5633ce" containerID="9f66f5c3fe36fdebd2739e1df14a07a42826524704e781efccdbe1172a13a4e3" exitCode=0
Jul 31 14:48:15.980628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:15.980245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerDied","Data":"9f66f5c3fe36fdebd2739e1df14a07a42826524704e781efccdbe1172a13a4e3"}
Jul 31 14:48:16.986506 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:16.986282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" event={"ID":"5304b752-64c8-4148-9ba4-ff4c6c5633ce","Type":"ContainerStarted","Data":"9e6ce2ae24aedf981e1a6b7d90a60c2c69b76d0af4d2ae6ff65fb5fc428b58c6"}
Jul 31 14:48:16.987751 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:16.987725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5qq4" event={"ID":"dbd836fa-93a0-4c34-a332-c3606ef36615","Type":"ContainerStarted","Data":"70226462d90ca51312f69266afa3301f7deddf3ece88f68ee648e13972130331"}
Jul 31 14:48:17.038939 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:17.038893    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g5qq4" podStartSLOduration=2.161437087 podStartE2EDuration="5.038878758s" totalImagesPullingTime="2.877441671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:17.038470252 +0000 UTC m=+39.750444781" watchObservedRunningTime="2026-07-31 14:48:17.038878758 +0000 UTC m=+39.750853285"
Jul 31 14:48:17.039096 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:17.039033    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h9nkf" podStartSLOduration=8.488841072 podStartE2EDuration="39.039021756s" totalImagesPullingTime="30.550180684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:17.019325851 +0000 UTC m=+39.731300398" watchObservedRunningTime="2026-07-31 14:48:17.039021756 +0000 UTC m=+39.750996288"
Jul 31 14:48:17.158166 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:17.158087    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log" limit=1024
Jul 31 14:48:17.254261 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:17.254227    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log" limit=1024
Jul 31 14:48:17.354775 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:17.354744    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log" limit=1024
Jul 31 14:48:19.891306 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:19.891270    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:19.891641 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:19.891385    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:19.891641 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:19.891396    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:19.891641 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:19.891456    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:27.891439288 +0000 UTC m=+50.603413815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:19.991502 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:19.991472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:19.991625 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:19.991608    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:19.991663 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:19.991658    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:27.991646142 +0000 UTC m=+50.703620650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:27.713408 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.713373    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"]
Jul 31 14:48:27.745083 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.745057    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"]
Jul 31 14:48:27.745229 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.745142    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.748916 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.748893    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 31 14:48:27.749341 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.749325    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 31 14:48:27.750348 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.750325    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ftxnh\""
Jul 31 14:48:27.750432 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.750394    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 31 14:48:27.750483 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.750435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 31 14:48:27.816839 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.816791    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-lwjqb"]
Jul 31 14:48:27.843009 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.842984    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-lwjqb"]
Jul 31 14:48:27.843117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.843059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.846006 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.845972    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 31 14:48:27.846006 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.845980    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9gsvd\""
Jul 31 14:48:27.846200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846054    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 31 14:48:27.846200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846072    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 31 14:48:27.846200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846100    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 31 14:48:27.846718 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/131917c6-74ad-4b9c-8fbe-f75a9e57193c-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.846849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.846908 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.846871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv7bh\" (UniqueName: \"kubernetes.io/projected/131917c6-74ad-4b9c-8fbe-f75a9e57193c-kube-api-access-pv7bh\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.851203 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.851187    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 31 14:48:27.916644 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.916618    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-n2v68"]
Jul 31 14:48:27.939001 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.938977    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"]
Jul 31 14:48:27.939124 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.939093    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:27.941964 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.941946    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 31 14:48:27.942062 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.941997    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 31 14:48:27.942520 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.942506    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-v8mff\""
Jul 31 14:48:27.943110 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.943095    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 31 14:48:27.943365 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.943349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:48:27.947099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-tmp\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947186 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:27.947186 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.947186 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947166    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-trusted-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947187    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv7bh\" (UniqueName: \"kubernetes.io/projected/131917c6-74ad-4b9c-8fbe-f75a9e57193c-kube-api-access-pv7bh\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947202    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-service-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/131917c6-74ad-4b9c-8fbe-f75a9e57193c-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:27.947246    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6802a36-92bb-4e6a-a8ad-d86898381e2c-serving-cert\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:27.947249    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:27.947305    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:28.4472901 +0000 UTC m=+51.159264608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:27.947314 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:27.947313    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:27.947636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-snapshots\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947636 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:27.947357    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.947345367 +0000 UTC m=+66.659319874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:27.947636 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwk6j\" (UniqueName: \"kubernetes.io/projected/a6802a36-92bb-4e6a-a8ad-d86898381e2c-kube-api-access-wwk6j\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:27.947868 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.947845    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/131917c6-74ad-4b9c-8fbe-f75a9e57193c-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.948930 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.948561    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 31 14:48:27.958418 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.958395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv7bh\" (UniqueName: \"kubernetes.io/projected/131917c6-74ad-4b9c-8fbe-f75a9e57193c-kube-api-access-pv7bh\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:27.962926 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.962906    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"]
Jul 31 14:48:27.963020 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.963006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:27.966041 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.965993    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 31 14:48:27.966041 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.965992    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-stg58\""
Jul 31 14:48:27.966173 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.966071    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:48:27.966279 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.966266    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 31 14:48:27.987136 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.987112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-n2v68"]
Jul 31 14:48:27.987234 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.987146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"]
Jul 31 14:48:27.987234 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.987176    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"]
Jul 31 14:48:27.987336 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.987262    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:27.989895 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.989878    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 31 14:48:27.989999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.989892    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 31 14:48:27.989999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.989927    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fpzzn\""
Jul 31 14:48:27.989999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.989952    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 31 14:48:27.989999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:27.989881    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:48:28.031465 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.031413    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"]
Jul 31 14:48:28.047706 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6802a36-92bb-4e6a-a8ad-d86898381e2c-serving-cert\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.047845 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhbg\" (UniqueName: \"kubernetes.io/projected/068a23c8-8c6f-471d-9847-d938b1348cb7-kube-api-access-rdhbg\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.047845 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e00e5e8b-4f02-406d-a8aa-3966ed31acae-serving-cert\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.047845 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:28.047987 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-service-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.047987 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3aad374-423d-4a82-8a0d-e2947ecc6168-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.047987 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.047915    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:28.047987 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.047929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-tmp\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.047987 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.047968    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:44.047950782 +0000 UTC m=+66.759925312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:28.048224 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-trusted-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.048224 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-trusted-ca\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.048224 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048141    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-config\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.048224 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqvv6\" (UniqueName: \"kubernetes.io/projected/b3aad374-423d-4a82-8a0d-e2947ecc6168-kube-api-access-wqvv6\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.048420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048240    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-snapshots\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.048420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwk6j\" (UniqueName: \"kubernetes.io/projected/a6802a36-92bb-4e6a-a8ad-d86898381e2c-kube-api-access-wwk6j\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.048420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-tmp\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.048420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048392    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.048605 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8996g\" (UniqueName: \"kubernetes.io/projected/e00e5e8b-4f02-406d-a8aa-3966ed31acae-kube-api-access-8996g\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.048605 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048489    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3aad374-423d-4a82-8a0d-e2947ecc6168-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.048605 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048506    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-service-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.048947 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048921    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6802a36-92bb-4e6a-a8ad-d86898381e2c-trusted-ca-bundle\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.049064 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.048978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6802a36-92bb-4e6a-a8ad-d86898381e2c-snapshots\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.050697 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.050674    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6802a36-92bb-4e6a-a8ad-d86898381e2c-serving-cert\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.053334 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.053306    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"]
Jul 31 14:48:28.053479 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.053425    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.056132 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.056111    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 31 14:48:28.056427 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.056403    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mzgmv\""
Jul 31 14:48:28.056533 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.056457    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 31 14:48:28.056606 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.056590    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 31 14:48:28.056649 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.056609    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:48:28.069371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.069345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwk6j\" (UniqueName: \"kubernetes.io/projected/a6802a36-92bb-4e6a-a8ad-d86898381e2c-kube-api-access-wwk6j\") pod \"insights-operator-77696685b-lwjqb\" (UID: \"a6802a36-92bb-4e6a-a8ad-d86898381e2c\") " pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.073482 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.073463    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-587cb6667d-857jd"]
Jul 31 14:48:28.073592 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.073578    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"
Jul 31 14:48:28.077309 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.077287    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 31 14:48:28.077309 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.077304    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-82fzj\""
Jul 31 14:48:28.077547 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.077533    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 31 14:48:28.099434 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.099417    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"]
Jul 31 14:48:28.099526 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.099443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"]
Jul 31 14:48:28.099526 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.099457    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-587cb6667d-857jd"]
Jul 31 14:48:28.099526 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.099517    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.103142 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.103126    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9s2qq\""
Jul 31 14:48:28.103591 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.103570    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 31 14:48:28.103679 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.103574    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 31 14:48:28.104032 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.104008    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 14:48:28.104149 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.104131    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 31 14:48:28.104225 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.104181    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 31 14:48:28.104307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.104286    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 14:48:28.149449 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149424    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3aad374-423d-4a82-8a0d-e2947ecc6168-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.149572 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhbg\" (UniqueName: \"kubernetes.io/projected/068a23c8-8c6f-471d-9847-d938b1348cb7-kube-api-access-rdhbg\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.149572 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e00e5e8b-4f02-406d-a8aa-3966ed31acae-serving-cert\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.149572 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-default-certificate\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.149725 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3aad374-423d-4a82-8a0d-e2947ecc6168-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.149802 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.149872 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60deba42-8140-4e36-bfee-4b289d360099-config\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.149872 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-trusted-ca\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.149970 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sncfr\" (UniqueName: \"kubernetes.io/projected/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-kube-api-access-sncfr\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.149970 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-config\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.149970 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqvv6\" (UniqueName: \"kubernetes.io/projected/b3aad374-423d-4a82-8a0d-e2947ecc6168-kube-api-access-wqvv6\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.150100 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.149989    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzlv4\" (UniqueName: \"kubernetes.io/projected/60deba42-8140-4e36-bfee-4b289d360099-kube-api-access-wzlv4\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.150100 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-stats-auth\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.150100 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.150241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60deba42-8140-4e36-bfee-4b289d360099-serving-cert\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.150241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.150241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8996g\" (UniqueName: \"kubernetes.io/projected/e00e5e8b-4f02-406d-a8aa-3966ed31acae-kube-api-access-8996g\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.150241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4z7c\" (UniqueName: \"kubernetes.io/projected/aec941e5-d6ec-4f49-a23c-25291523162f-kube-api-access-w4z7c\") pod \"volume-data-source-validator-5b8776669-9rhl5\" (UID: \"aec941e5-d6ec-4f49-a23c-25291523162f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"
Jul 31 14:48:28.150435 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3aad374-423d-4a82-8a0d-e2947ecc6168-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.150435 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.150431    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 14:48:28.150529 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.150485    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls podName:068a23c8-8c6f-471d-9847-d938b1348cb7 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:28.65047348 +0000 UTC m=+51.362447987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-7kdlj" (UID: "068a23c8-8c6f-471d-9847-d938b1348cb7") : secret "samples-operator-tls" not found
Jul 31 14:48:28.150681 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150662    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-config\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.150798 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150776    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e00e5e8b-4f02-406d-a8aa-3966ed31acae-trusted-ca\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.150928 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.150913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-lwjqb"
Jul 31 14:48:28.151929 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.151908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3aad374-423d-4a82-8a0d-e2947ecc6168-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.152041 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.152026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e00e5e8b-4f02-406d-a8aa-3966ed31acae-serving-cert\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.166057 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.166012    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhbg\" (UniqueName: \"kubernetes.io/projected/068a23c8-8c6f-471d-9847-d938b1348cb7-kube-api-access-rdhbg\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.168338 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.168318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqvv6\" (UniqueName: \"kubernetes.io/projected/b3aad374-423d-4a82-8a0d-e2947ecc6168-kube-api-access-wqvv6\") pod \"kube-storage-version-migrator-operator-7878f4b88c-bxpvh\" (UID: \"b3aad374-423d-4a82-8a0d-e2947ecc6168\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.168595 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.168576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8996g\" (UniqueName: \"kubernetes.io/projected/e00e5e8b-4f02-406d-a8aa-3966ed31acae-kube-api-access-8996g\") pod \"console-operator-5ccd6b445f-n2v68\" (UID: \"e00e5e8b-4f02-406d-a8aa-3966ed31acae\") " pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.247090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.247056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:28.251664 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.251799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60deba42-8140-4e36-bfee-4b289d360099-config\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.251799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sncfr\" (UniqueName: \"kubernetes.io/projected/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-kube-api-access-sncfr\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.251799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzlv4\" (UniqueName: \"kubernetes.io/projected/60deba42-8140-4e36-bfee-4b289d360099-kube-api-access-wzlv4\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.251799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-stats-auth\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.251799 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.251852    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:28.751823408 +0000 UTC m=+51.463797932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.251900    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.251911    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60deba42-8140-4e36-bfee-4b289d360099-serving-cert\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.251947    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:28.751928471 +0000 UTC m=+51.463902978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.252015    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4z7c\" (UniqueName: \"kubernetes.io/projected/aec941e5-d6ec-4f49-a23c-25291523162f-kube-api-access-w4z7c\") pod \"volume-data-source-validator-5b8776669-9rhl5\" (UID: \"aec941e5-d6ec-4f49-a23c-25291523162f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"
Jul 31 14:48:28.252067 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.252042    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-default-certificate\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.252907 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.252885    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/60deba42-8140-4e36-bfee-4b289d360099-config\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.254105 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.254080    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60deba42-8140-4e36-bfee-4b289d360099-serving-cert\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.254369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.254352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-stats-auth\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.254442 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.254369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-default-certificate\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.268212 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.268146    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4z7c\" (UniqueName: \"kubernetes.io/projected/aec941e5-d6ec-4f49-a23c-25291523162f-kube-api-access-w4z7c\") pod \"volume-data-source-validator-5b8776669-9rhl5\" (UID: \"aec941e5-d6ec-4f49-a23c-25291523162f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"
Jul 31 14:48:28.268366 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.268343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzlv4\" (UniqueName: \"kubernetes.io/projected/60deba42-8140-4e36-bfee-4b289d360099-kube-api-access-wzlv4\") pod \"service-ca-operator-784f97bcf6-5tc6j\" (UID: \"60deba42-8140-4e36-bfee-4b289d360099\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.274154 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.274133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sncfr\" (UniqueName: \"kubernetes.io/projected/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-kube-api-access-sncfr\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.287573 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.287550    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-lwjqb"]
Jul 31 14:48:28.290608 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:28.290522    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6802a36_92bb_4e6a_a8ad_d86898381e2c.slice/crio-0dace48948e9791df0f66a74e5eca8239d626999d1f9813928d94cd87bebf3bd WatchSource:0}: Error finding container 0dace48948e9791df0f66a74e5eca8239d626999d1f9813928d94cd87bebf3bd: Status 404 returned error can't find the container with id 0dace48948e9791df0f66a74e5eca8239d626999d1f9813928d94cd87bebf3bd
Jul 31 14:48:28.295439 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.295414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"
Jul 31 14:48:28.365565 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.365088    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"
Jul 31 14:48:28.387451 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.385018    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"
Jul 31 14:48:28.440153 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.440068    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-n2v68"]
Jul 31 14:48:28.442571 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:28.442543    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00e5e8b_4f02_406d_a8aa_3966ed31acae.slice/crio-0415b8469e6d5593d60c0d37f2b59dfce264c11b2f0273916f52b1f0f8a90970 WatchSource:0}: Error finding container 0415b8469e6d5593d60c0d37f2b59dfce264c11b2f0273916f52b1f0f8a90970: Status 404 returned error can't find the container with id 0415b8469e6d5593d60c0d37f2b59dfce264c11b2f0273916f52b1f0f8a90970
Jul 31 14:48:28.455343 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.455109    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:28.455343 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.455259    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:28.455343 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.455324    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:29.455302537 +0000 UTC m=+52.167277064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:28.466466 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.466278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh"]
Jul 31 14:48:28.469061 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:28.469019    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3aad374_423d_4a82_8a0d_e2947ecc6168.slice/crio-8b37b9f7a310a6fb31ada611be890ef4ba4b876f5c30c1d3f628c5f7679e76ae WatchSource:0}: Error finding container 8b37b9f7a310a6fb31ada611be890ef4ba4b876f5c30c1d3f628c5f7679e76ae: Status 404 returned error can't find the container with id 8b37b9f7a310a6fb31ada611be890ef4ba4b876f5c30c1d3f628c5f7679e76ae
Jul 31 14:48:28.529277 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.529150    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j"]
Jul 31 14:48:28.532065 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:28.532039    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60deba42_8140_4e36_bfee_4b289d360099.slice/crio-23f7ac8262b2cc4863580c0678f5dbf49b14ca7d3298ddc4c84cbefe5eee91f3 WatchSource:0}: Error finding container 23f7ac8262b2cc4863580c0678f5dbf49b14ca7d3298ddc4c84cbefe5eee91f3: Status 404 returned error can't find the container with id 23f7ac8262b2cc4863580c0678f5dbf49b14ca7d3298ddc4c84cbefe5eee91f3
Jul 31 14:48:28.544626 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.544604    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5"]
Jul 31 14:48:28.547083 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:28.547057    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec941e5_d6ec_4f49_a23c_25291523162f.slice/crio-c730a3d805ec8f497a422b4aa45e430d2cb1bcc1f16b2f884f11918d613f24d5 WatchSource:0}: Error finding container c730a3d805ec8f497a422b4aa45e430d2cb1bcc1f16b2f884f11918d613f24d5: Status 404 returned error can't find the container with id c730a3d805ec8f497a422b4aa45e430d2cb1bcc1f16b2f884f11918d613f24d5
Jul 31 14:48:28.657283 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.657258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:28.657443 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.657421    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 14:48:28.657503 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.657497    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls podName:068a23c8-8c6f-471d-9847-d938b1348cb7 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:29.657476788 +0000 UTC m=+52.369451307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-7kdlj" (UID: "068a23c8-8c6f-471d-9847-d938b1348cb7") : secret "samples-operator-tls" not found
Jul 31 14:48:28.758149 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.758075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.758149 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:28.758123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:28.758491 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.758247    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:29.758228131 +0000 UTC m=+52.470202655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:28.758491 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.758274    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:28.758491 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:28.758309    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:29.758298702 +0000 UTC m=+52.470273227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:29.018657 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.018581    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" event={"ID":"e00e5e8b-4f02-406d-a8aa-3966ed31acae","Type":"ContainerStarted","Data":"0415b8469e6d5593d60c0d37f2b59dfce264c11b2f0273916f52b1f0f8a90970"}
Jul 31 14:48:29.019768 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.019742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5" event={"ID":"aec941e5-d6ec-4f49-a23c-25291523162f","Type":"ContainerStarted","Data":"c730a3d805ec8f497a422b4aa45e430d2cb1bcc1f16b2f884f11918d613f24d5"}
Jul 31 14:48:29.020926 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.020895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j" event={"ID":"60deba42-8140-4e36-bfee-4b289d360099","Type":"ContainerStarted","Data":"23f7ac8262b2cc4863580c0678f5dbf49b14ca7d3298ddc4c84cbefe5eee91f3"}
Jul 31 14:48:29.022176 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.022141    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-lwjqb" event={"ID":"a6802a36-92bb-4e6a-a8ad-d86898381e2c","Type":"ContainerStarted","Data":"0dace48948e9791df0f66a74e5eca8239d626999d1f9813928d94cd87bebf3bd"}
Jul 31 14:48:29.023361 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.023336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh" event={"ID":"b3aad374-423d-4a82-8a0d-e2947ecc6168","Type":"ContainerStarted","Data":"8b37b9f7a310a6fb31ada611be890ef4ba4b876f5c30c1d3f628c5f7679e76ae"}
Jul 31 14:48:29.464427 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.464388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:29.464618 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.464597    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:29.464689 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.464662    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:31.464643468 +0000 UTC m=+54.176617980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:29.666297 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.665588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:29.666297 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.665896    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 14:48:29.666297 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.665963    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls podName:068a23c8-8c6f-471d-9847-d938b1348cb7 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:31.665944166 +0000 UTC m=+54.377918679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-7kdlj" (UID: "068a23c8-8c6f-471d-9847-d938b1348cb7") : secret "samples-operator-tls" not found
Jul 31 14:48:29.768017 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.767178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:29.768017 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:29.767259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:29.768017 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.767463    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:29.768017 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.767531    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:31.767512289 +0000 UTC m=+54.479486802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:29.768017 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:29.767930    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:31.767919023 +0000 UTC m=+54.479893544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:31.483328 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:31.483288    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:31.483731 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.483461    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:31.483731 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.483530    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.483515446 +0000 UTC m=+58.195489956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:31.684513 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:31.684476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:31.684687 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.684647    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 14:48:31.684761 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.684729    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls podName:068a23c8-8c6f-471d-9847-d938b1348cb7 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.684710948 +0000 UTC m=+58.396685455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-7kdlj" (UID: "068a23c8-8c6f-471d-9847-d938b1348cb7") : secret "samples-operator-tls" not found
Jul 31 14:48:31.785584 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:31.785500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:31.785826 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.785656    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:31.785826 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:31.785677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:31.785826 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.785725    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.78570443 +0000 UTC m=+58.497678938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:31.785993 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:31.785842    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.785821709 +0000 UTC m=+58.497796221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:33.036893 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.036864    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/0.log"
Jul 31 14:48:33.037324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.036905    2566 generic.go:356] "Generic (PLEG): container finished" podID="e00e5e8b-4f02-406d-a8aa-3966ed31acae" containerID="ff9b888b2586e0c010da057670a42c5b12a8016fe46360420663dcb81e3627e9" exitCode=255
Jul 31 14:48:33.037324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.036996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" event={"ID":"e00e5e8b-4f02-406d-a8aa-3966ed31acae","Type":"ContainerDied","Data":"ff9b888b2586e0c010da057670a42c5b12a8016fe46360420663dcb81e3627e9"}
Jul 31 14:48:33.037324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.037187    2566 scope.go:117] "RemoveContainer" containerID="ff9b888b2586e0c010da057670a42c5b12a8016fe46360420663dcb81e3627e9"
Jul 31 14:48:33.040766 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.040733    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5" event={"ID":"aec941e5-d6ec-4f49-a23c-25291523162f","Type":"ContainerStarted","Data":"8d2b54dd73f314ddde2695520a90d7957060f2f2da24b974f6dce4faea9b8d74"}
Jul 31 14:48:33.042745 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.042314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j" event={"ID":"60deba42-8140-4e36-bfee-4b289d360099","Type":"ContainerStarted","Data":"8a354aee6feb3253c6d197205187e55cd129267fd61db29ab579163538cc5e9a"}
Jul 31 14:48:33.044278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.044251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-lwjqb" event={"ID":"a6802a36-92bb-4e6a-a8ad-d86898381e2c","Type":"ContainerStarted","Data":"908f0b4e79a12334b60b0d5f5de8b445393caa995e0f32861a99048e073c767b"}
Jul 31 14:48:33.045748 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.045725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh" event={"ID":"b3aad374-423d-4a82-8a0d-e2947ecc6168","Type":"ContainerStarted","Data":"af06978eab882b1167f09da03f8a6b31b2763545c065275deb23ba57ff9bde19"}
Jul 31 14:48:33.087942 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.087891    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-lwjqb" podStartSLOduration=2.180881243 podStartE2EDuration="6.087871887s" totalImagesPullingTime="3.906990644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:33.086487956 +0000 UTC m=+55.798462485" watchObservedRunningTime="2026-07-31 14:48:33.087871887 +0000 UTC m=+55.799846414"
Jul 31 14:48:33.108481 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.108420    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh" podStartSLOduration=2.376523697 podStartE2EDuration="6.108399198s" totalImagesPullingTime="3.731875501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:33.106832834 +0000 UTC m=+55.818807361" watchObservedRunningTime="2026-07-31 14:48:33.108399198 +0000 UTC m=+55.820373727"
Jul 31 14:48:33.134567 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.134511    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j" podStartSLOduration=1.462954922 podStartE2EDuration="5.134478683s" totalImagesPullingTime="3.671523761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:33.132997496 +0000 UTC m=+55.844972035" watchObservedRunningTime="2026-07-31 14:48:33.134478683 +0000 UTC m=+55.846453212"
Jul 31 14:48:33.160531 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.160415    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-9rhl5" podStartSLOduration=1.5114430730000001 podStartE2EDuration="5.160399963s" totalImagesPullingTime="3.64895689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:33.159152227 +0000 UTC m=+55.871126757" watchObservedRunningTime="2026-07-31 14:48:33.160399963 +0000 UTC m=+55.872374492"
Jul 31 14:48:33.623708 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.623677    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m8g29"]
Jul 31 14:48:33.625932 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.625913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.639248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.639229    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gfnvf\""
Jul 31 14:48:33.641276 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.641260    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 31 14:48:33.641995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.641982    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 31 14:48:33.678619 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.678596    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8g29"]
Jul 31 14:48:33.702299 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.702267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.702428 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.702307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.702428 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.702397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f1d198-7f7a-4cea-b8fd-1806ac828fde-crio-socket\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.702511 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.702471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f1d198-7f7a-4cea-b8fd-1806ac828fde-data-volume\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.702511 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.702508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87mg\" (UniqueName: \"kubernetes.io/projected/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-api-access-c87mg\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803064 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f1d198-7f7a-4cea-b8fd-1806ac828fde-crio-socket\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803211 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f1d198-7f7a-4cea-b8fd-1806ac828fde-crio-socket\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803211 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f1d198-7f7a-4cea-b8fd-1806ac828fde-data-volume\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803280 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87mg\" (UniqueName: \"kubernetes.io/projected/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-api-access-c87mg\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803366 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803406 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803472 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.803452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f1d198-7f7a-4cea-b8fd-1806ac828fde-data-volume\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:33.803525 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:33.803486    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 14:48:33.803525 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:33.803507    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:33.803591 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:33.803553    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:34.303533703 +0000 UTC m=+57.015508214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : configmap "kube-rbac-proxy" not found
Jul 31 14:48:33.803591 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:33.803580    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:34.303560456 +0000 UTC m=+57.015534970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:33.812995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:33.812974    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87mg\" (UniqueName: \"kubernetes.io/projected/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-api-access-c87mg\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:34.050946 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.050919    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/1.log"
Jul 31 14:48:34.051374 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.051359    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/0.log"
Jul 31 14:48:34.051413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.051392    2566 generic.go:356] "Generic (PLEG): container finished" podID="e00e5e8b-4f02-406d-a8aa-3966ed31acae" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d" exitCode=255
Jul 31 14:48:34.051478 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.051453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" event={"ID":"e00e5e8b-4f02-406d-a8aa-3966ed31acae","Type":"ContainerDied","Data":"b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"}
Jul 31 14:48:34.051549 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.051535    2566 scope.go:117] "RemoveContainer" containerID="ff9b888b2586e0c010da057670a42c5b12a8016fe46360420663dcb81e3627e9"
Jul 31 14:48:34.051744 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.051722    2566 scope.go:117] "RemoveContainer" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"
Jul 31 14:48:34.051955 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:34.051927    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-n2v68_openshift-console-operator(e00e5e8b-4f02-406d-a8aa-3966ed31acae)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podUID="e00e5e8b-4f02-406d-a8aa-3966ed31acae"
Jul 31 14:48:34.308782 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.308698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:34.308782 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.308740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:34.308975 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:34.308860    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 14:48:34.308975 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:34.308882    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:34.308975 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:34.308913    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.308899468 +0000 UTC m=+58.020873978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : configmap "kube-rbac-proxy" not found
Jul 31 14:48:34.308975 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:34.308944    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:35.308930452 +0000 UTC m=+58.020904964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:34.954569 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:34.954544    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krg8d_f634de43-ec15-469d-95e8-932e660ca0ab/dns-node-resolver/0.log"
Jul 31 14:48:35.055502 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.055476    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/1.log"
Jul 31 14:48:35.055858 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.055829    2566 scope.go:117] "RemoveContainer" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"
Jul 31 14:48:35.055994 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.055976    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-n2v68_openshift-console-operator(e00e5e8b-4f02-406d-a8aa-3966ed31acae)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podUID="e00e5e8b-4f02-406d-a8aa-3966ed31acae"
Jul 31 14:48:35.317371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.317278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:35.317371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.317323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:35.317558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.317448    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:35.317558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.317480    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 14:48:35.317558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.317515    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:37.31749983 +0000 UTC m=+60.029474340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:35.317558 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.317529    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:37.317522864 +0000 UTC m=+60.029497371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : configmap "kube-rbac-proxy" not found
Jul 31 14:48:35.519248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.519214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:35.519409 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.519359    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:35.519449 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.519423    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.5194076 +0000 UTC m=+66.231382107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:35.721023 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.720983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:35.721159 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.721141    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 14:48:35.721205 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.721195    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls podName:068a23c8-8c6f-471d-9847-d938b1348cb7 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.721182014 +0000 UTC m=+66.433156521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-7kdlj" (UID: "068a23c8-8c6f-471d-9847-d938b1348cb7") : secret "samples-operator-tls" not found
Jul 31 14:48:35.754516 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.754492    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bhw7_fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27/node-ca/0.log"
Jul 31 14:48:35.822200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.822175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:35.822327 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:35.822249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:35.822380 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.822335    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.82231942 +0000 UTC m=+66.534293927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:35.822425 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.822410    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:35.822533 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:35.822457    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:43.822442808 +0000 UTC m=+66.534417324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:36.555091 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:36.555059    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5qq4_dbd836fa-93a0-4c34-a332-c3606ef36615/serve-healthcheck-canary/0.log"
Jul 31 14:48:36.969384 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:36.969361    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksbpc"
Jul 31 14:48:37.335604 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:37.335509    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:37.335604 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:37.335558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:37.335853 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:37.335640    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 14:48:37.335853 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:37.335671    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:37.335853 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:37.335707    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:41.335688893 +0000 UTC m=+64.047663400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : configmap "kube-rbac-proxy" not found
Jul 31 14:48:37.335853 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:37.335728    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:41.335715901 +0000 UTC m=+64.047690408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:38.247973 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:38.247931    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:38.248404 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:38.247998    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:38.248404 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:38.248298    2566 scope.go:117] "RemoveContainer" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"
Jul 31 14:48:38.248478 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:38.248428    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-n2v68_openshift-console-operator(e00e5e8b-4f02-406d-a8aa-3966ed31acae)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podUID="e00e5e8b-4f02-406d-a8aa-3966ed31acae"
Jul 31 14:48:41.371720 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:41.371681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:41.371720 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:41.371723    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:41.372191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:41.371860    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:41.372191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:41.371874    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 14:48:41.372191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:41.371931    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:49.371907824 +0000 UTC m=+72.083882331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:41.372191 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:41.371947    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:48:49.371940157 +0000 UTC m=+72.083914664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : configmap "kube-rbac-proxy" not found
Jul 31 14:48:43.494142 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.494103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:48:43.497101 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.497081    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 14:48:43.504785 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.504766    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 31 14:48:43.504868 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.504841    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs podName:1b29e6e9-f8e0-451e-b658-cbf0ed5ab089 nodeName:}" failed. No retries permitted until 2026-07-31 14:49:47.504821991 +0000 UTC m=+130.216796512 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs") pod "network-metrics-daemon-n8x7h" (UID: "1b29e6e9-f8e0-451e-b658-cbf0ed5ab089") : secret "metrics-daemon-secret" not found
Jul 31 14:48:43.595004 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.594970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:43.595136 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.595033    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:43.595136 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.595067    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:43.595206 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.595124    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:43.595206 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.595196    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls podName:131917c6-74ad-4b9c-8fbe-f75a9e57193c nodeName:}" failed. No retries permitted until 2026-07-31 14:48:59.595178136 +0000 UTC m=+82.307152644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7pkb4" (UID: "131917c6-74ad-4b9c-8fbe-f75a9e57193c") : secret "cluster-monitoring-operator-tls" not found
Jul 31 14:48:43.598983 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.598968    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 14:48:43.599176 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.599162    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 14:48:43.607879 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.607857    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9bc66d4e-9a34-4922-b231-cf1c4793428a-original-pull-secret\") pod \"global-pull-secret-syncer-vdqg9\" (UID: \"9bc66d4e-9a34-4922-b231-cf1c4793428a\") " pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:43.612657 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.612635    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 14:48:43.617913 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.617894    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n67vn\" (UniqueName: \"kubernetes.io/projected/4f3ebed1-1af2-4f49-b77e-14efe59b929c-kube-api-access-n67vn\") pod \"network-check-target-7vzl5\" (UID: \"4f3ebed1-1af2-4f49-b77e-14efe59b929c\") " pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:43.722959 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.722928    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2kc8\""
Jul 31 14:48:43.727363 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.727345    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdqg9"
Jul 31 14:48:43.730469 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.730454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:43.797384 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.797293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:43.799686 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.799663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/068a23c8-8c6f-471d-9847-d938b1348cb7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-7kdlj\" (UID: \"068a23c8-8c6f-471d-9847-d938b1348cb7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:43.874216 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.874188    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-stg58\""
Jul 31 14:48:43.881978 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.881949    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"
Jul 31 14:48:43.898454 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.898430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:43.898570 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.898545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:43.898613 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.898567    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 14:48:43.898651 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.898621    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:59.898606763 +0000 UTC m=+82.610581270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : secret "router-metrics-certs-default" not found
Jul 31 14:48:43.898696 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.898650    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle podName:2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179 nodeName:}" failed. No retries permitted until 2026-07-31 14:48:59.898639201 +0000 UTC m=+82.610613756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle") pod "router-default-587cb6667d-857jd" (UID: "2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179") : configmap references non-existent config key: service-ca.crt
Jul 31 14:48:43.907286 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.907265    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdqg9"]
Jul 31 14:48:43.917024 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:43.917003    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc66d4e_9a34_4922_b231_cf1c4793428a.slice/crio-9303e3d97d63cfbd82955c9db88a709bd6e96118b02741fb29186dfd02666d50 WatchSource:0}: Error finding container 9303e3d97d63cfbd82955c9db88a709bd6e96118b02741fb29186dfd02666d50: Status 404 returned error can't find the container with id 9303e3d97d63cfbd82955c9db88a709bd6e96118b02741fb29186dfd02666d50
Jul 31 14:48:43.920056 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.919960    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7vzl5"]
Jul 31 14:48:43.921898 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:43.921873    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3ebed1_1af2_4f49_b77e_14efe59b929c.slice/crio-f7270fdd15f3b14a9f16a003c00d0d7ab521f442631b16c89e5b490e0ad8d8da WatchSource:0}: Error finding container f7270fdd15f3b14a9f16a003c00d0d7ab521f442631b16c89e5b490e0ad8d8da: Status 404 returned error can't find the container with id f7270fdd15f3b14a9f16a003c00d0d7ab521f442631b16c89e5b490e0ad8d8da
Jul 31 14:48:43.999371 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:43.999344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:48:43.999535 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.999496    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 14:48:43.999535 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.999511    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4655fff-5zqmc: secret "image-registry-tls" not found
Jul 31 14:48:43.999655 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:43.999565    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls podName:70c5dbd2-f468-4343-976d-0a117b21bd72 nodeName:}" failed. No retries permitted until 2026-07-31 14:49:15.999550311 +0000 UTC m=+98.711524838 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls") pod "image-registry-6f4655fff-5zqmc" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72") : secret "image-registry-tls" not found
Jul 31 14:48:44.045468 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:44.045445    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj"]
Jul 31 14:48:44.082313 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:44.082283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdqg9" event={"ID":"9bc66d4e-9a34-4922-b231-cf1c4793428a","Type":"ContainerStarted","Data":"9303e3d97d63cfbd82955c9db88a709bd6e96118b02741fb29186dfd02666d50"}
Jul 31 14:48:44.083249 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:44.083220    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7vzl5" event={"ID":"4f3ebed1-1af2-4f49-b77e-14efe59b929c","Type":"ContainerStarted","Data":"f7270fdd15f3b14a9f16a003c00d0d7ab521f442631b16c89e5b490e0ad8d8da"}
Jul 31 14:48:44.099760 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:44.099739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:48:44.099910 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:44.099893    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 14:48:44.099961 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:44.099956    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls podName:baca90d9-9cb5-43f1-a5b5-47826d395d31 nodeName:}" failed. No retries permitted until 2026-07-31 14:49:16.099942539 +0000 UTC m=+98.811917046 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls") pod "dns-default-n2xrq" (UID: "baca90d9-9cb5-43f1-a5b5-47826d395d31") : secret "dns-default-metrics-tls" not found
Jul 31 14:48:45.090411 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:45.090330    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj" event={"ID":"068a23c8-8c6f-471d-9847-d938b1348cb7","Type":"ContainerStarted","Data":"f7fb7009e63640de6ac51f772f16fc63b5e057160af7e75beffd3272cd31f5fc"}
Jul 31 14:48:49.105392 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.105353    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdqg9" event={"ID":"9bc66d4e-9a34-4922-b231-cf1c4793428a","Type":"ContainerStarted","Data":"2608fdd499eb98db2e700aef1ced5655509ec44923402da7ede756ec6c1ab72a"}
Jul 31 14:48:49.106789 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.106761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7vzl5" event={"ID":"4f3ebed1-1af2-4f49-b77e-14efe59b929c","Type":"ContainerStarted","Data":"350eb211536a1f2642a3781de28353a7de8a89226b49f93b43663fc364f7d1c3"}
Jul 31 14:48:49.106914 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.106837    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:48:49.108467 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.108441    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj" event={"ID":"068a23c8-8c6f-471d-9847-d938b1348cb7","Type":"ContainerStarted","Data":"83be93e7fe10565e772fc0e80e0c34eb8401f7580fae5c528e952395b6e325d8"}
Jul 31 14:48:49.108586 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.108483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj" event={"ID":"068a23c8-8c6f-471d-9847-d938b1348cb7","Type":"ContainerStarted","Data":"27abb0728a3169a00fa758ef9101f5d17f218c9644b7c2243caf3716543b1db3"}
Jul 31 14:48:49.132742 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.132699    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vdqg9" podStartSLOduration=65.599547314 podStartE2EDuration="1m10.13268719s" totalImagesPullingTime="4.533139876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:49.132176446 +0000 UTC m=+71.844150975" watchObservedRunningTime="2026-07-31 14:48:49.13268719 +0000 UTC m=+71.844661719"
Jul 31 14:48:49.214430 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.214386    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-7kdlj" podStartSLOduration=17.876864133 podStartE2EDuration="22.214370956s" totalImagesPullingTime="4.337506823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:49.177549804 +0000 UTC m=+71.889524332" watchObservedRunningTime="2026-07-31 14:48:49.214370956 +0000 UTC m=+71.926345483"
Jul 31 14:48:49.214774 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.214755    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7vzl5" podStartSLOduration=66.693699206 podStartE2EDuration="1m11.214748617s" totalImagesPullingTime="4.521049411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:48:49.214250442 +0000 UTC m=+71.926224971" watchObservedRunningTime="2026-07-31 14:48:49.214748617 +0000 UTC m=+71.926723145"
Jul 31 14:48:49.449965 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.449938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:49.450105 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.450047    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:49.450144 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:49.450125    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:49.450176 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:49.450170    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls podName:09f1d198-7f7a-4cea-b8fd-1806ac828fde nodeName:}" failed. No retries permitted until 2026-07-31 14:49:05.450158835 +0000 UTC m=+88.162133345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8g29" (UID: "09f1d198-7f7a-4cea-b8fd-1806ac828fde") : secret "insights-runtime-extractor-tls" not found
Jul 31 14:48:49.450431 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:49.450412    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f1d198-7f7a-4cea-b8fd-1806ac828fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:48:50.808971 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:50.808940    2566 scope.go:117] "RemoveContainer" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"
Jul 31 14:48:51.116955 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.116877    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:48:51.117311 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.117294    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/1.log"
Jul 31 14:48:51.117378 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.117331    2566 generic.go:356] "Generic (PLEG): container finished" podID="e00e5e8b-4f02-406d-a8aa-3966ed31acae" containerID="3c7923ed42e2b0c7acd540f536d116ce2de320879f7cb9be37d010022a7af8e1" exitCode=255
Jul 31 14:48:51.117378 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.117366    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" event={"ID":"e00e5e8b-4f02-406d-a8aa-3966ed31acae","Type":"ContainerDied","Data":"3c7923ed42e2b0c7acd540f536d116ce2de320879f7cb9be37d010022a7af8e1"}
Jul 31 14:48:51.117435 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.117411    2566 scope.go:117] "RemoveContainer" containerID="b5f1a9079b725bdd99d55a1e094a4703338dcd840de100560402d3585fc56d8d"
Jul 31 14:48:51.117767 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:51.117746    2566 scope.go:117] "RemoveContainer" containerID="3c7923ed42e2b0c7acd540f536d116ce2de320879f7cb9be37d010022a7af8e1"
Jul 31 14:48:51.117992 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:51.117972    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-n2v68_openshift-console-operator(e00e5e8b-4f02-406d-a8aa-3966ed31acae)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podUID="e00e5e8b-4f02-406d-a8aa-3966ed31acae"
Jul 31 14:48:52.124125 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:52.124097    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:48:58.248169 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:58.248137    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:58.248598 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:58.248199    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:48:58.248598 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:58.248510    2566 scope.go:117] "RemoveContainer" containerID="3c7923ed42e2b0c7acd540f536d116ce2de320879f7cb9be37d010022a7af8e1"
Jul 31 14:48:58.248684 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:48:58.248663    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-n2v68_openshift-console-operator(e00e5e8b-4f02-406d-a8aa-3966ed31acae)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podUID="e00e5e8b-4f02-406d-a8aa-3966ed31acae"
Jul 31 14:48:59.637365 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.637327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:59.639623 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.639601    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/131917c6-74ad-4b9c-8fbe-f75a9e57193c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7pkb4\" (UID: \"131917c6-74ad-4b9c-8fbe-f75a9e57193c\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:59.857427 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.857393    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ftxnh\""
Jul 31 14:48:59.864063 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.864045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"
Jul 31 14:48:59.939668 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.939632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:59.939840 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.939719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:59.940295 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.940223    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-service-ca-bundle\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:59.942769 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.942746    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179-metrics-certs\") pod \"router-default-587cb6667d-857jd\" (UID: \"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179\") " pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:48:59.991549 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:48:59.991514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4"]
Jul 31 14:48:59.994098 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:48:59.994071    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131917c6_74ad_4b9c_8fbe_f75a9e57193c.slice/crio-8da714528558985b364c33e66f0b4443c17b688edfdb9a4e89857b8653573be5 WatchSource:0}: Error finding container 8da714528558985b364c33e66f0b4443c17b688edfdb9a4e89857b8653573be5: Status 404 returned error can't find the container with id 8da714528558985b364c33e66f0b4443c17b688edfdb9a4e89857b8653573be5
Jul 31 14:49:00.153522 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:00.153488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4" event={"ID":"131917c6-74ad-4b9c-8fbe-f75a9e57193c","Type":"ContainerStarted","Data":"8da714528558985b364c33e66f0b4443c17b688edfdb9a4e89857b8653573be5"}
Jul 31 14:49:00.211064 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:00.211039    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9s2qq\""
Jul 31 14:49:00.218646 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:00.218625    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:49:00.354184 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:00.354163    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-587cb6667d-857jd"]
Jul 31 14:49:00.356084 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:00.356059    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c39bfbf_b3d4_4aa9_8c0e_3c5a6ffac179.slice/crio-b3b089d49f65df15e387f247829bf29f277d617872b06a65bfdad19883c826c0 WatchSource:0}: Error finding container b3b089d49f65df15e387f247829bf29f277d617872b06a65bfdad19883c826c0: Status 404 returned error can't find the container with id b3b089d49f65df15e387f247829bf29f277d617872b06a65bfdad19883c826c0
Jul 31 14:49:01.158574 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:01.158526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-587cb6667d-857jd" event={"ID":"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179","Type":"ContainerStarted","Data":"ee07fc7d000bc38edd17e676cc62d4b4feaab63696924ac9e9c235d366d14d6a"}
Jul 31 14:49:01.159039 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:01.158588    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-587cb6667d-857jd" event={"ID":"2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179","Type":"ContainerStarted","Data":"b3b089d49f65df15e387f247829bf29f277d617872b06a65bfdad19883c826c0"}
Jul 31 14:49:01.180670 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:01.180601    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-587cb6667d-857jd" podStartSLOduration=33.180585447 podStartE2EDuration="33.180585447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:01.179361198 +0000 UTC m=+83.891335725" watchObservedRunningTime="2026-07-31 14:49:01.180585447 +0000 UTC m=+83.892559975"
Jul 31 14:49:01.219099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:01.219071    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:49:01.221979 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:01.221955    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:49:02.163132 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:02.163096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4" event={"ID":"131917c6-74ad-4b9c-8fbe-f75a9e57193c","Type":"ContainerStarted","Data":"2c3d7720aaa3de1045ee16161eae77daf0673020a99533e350147d3c3b0b73a4"}
Jul 31 14:49:02.163554 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:02.163346    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:49:02.164653 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:02.164634    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-587cb6667d-857jd"
Jul 31 14:49:02.183742 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:02.183033    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7pkb4" podStartSLOduration=33.132258736 podStartE2EDuration="35.183015127s" totalImagesPullingTime="2.050756391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:02.182415543 +0000 UTC m=+84.894390072" watchObservedRunningTime="2026-07-31 14:49:02.183015127 +0000 UTC m=+84.894989658"
Jul 31 14:49:05.484166 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:05.484122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:49:05.486418 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:05.486385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f1d198-7f7a-4cea-b8fd-1806ac828fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8g29\" (UID: \"09f1d198-7f7a-4cea-b8fd-1806ac828fde\") " pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:49:05.737998 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:05.737922    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gfnvf\""
Jul 31 14:49:05.745482 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:05.745458    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8g29"
Jul 31 14:49:05.878900 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:05.878873    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8g29"]
Jul 31 14:49:05.881173 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:05.881145    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f1d198_7f7a_4cea_b8fd_1806ac828fde.slice/crio-0b19353af2717db8da6b068d1a130f9eb9e3f3ad916ad35408a54a4a9047f1f4 WatchSource:0}: Error finding container 0b19353af2717db8da6b068d1a130f9eb9e3f3ad916ad35408a54a4a9047f1f4: Status 404 returned error can't find the container with id 0b19353af2717db8da6b068d1a130f9eb9e3f3ad916ad35408a54a4a9047f1f4
Jul 31 14:49:06.176821 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:06.176786    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8g29" event={"ID":"09f1d198-7f7a-4cea-b8fd-1806ac828fde","Type":"ContainerStarted","Data":"3bdc6c6b97697119c414c01b595a263d28562f0d0d189050b017ccb664b3a699"}
Jul 31 14:49:06.176979 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:06.176861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8g29" event={"ID":"09f1d198-7f7a-4cea-b8fd-1806ac828fde","Type":"ContainerStarted","Data":"0b19353af2717db8da6b068d1a130f9eb9e3f3ad916ad35408a54a4a9047f1f4"}
Jul 31 14:49:07.182762 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:07.182730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8g29" event={"ID":"09f1d198-7f7a-4cea-b8fd-1806ac828fde","Type":"ContainerStarted","Data":"dd7a1b037b8f9f00b3b81567fad4a6dca25804c88ac01abc766eb907776741a6"}
Jul 31 14:49:08.187607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:08.187570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8g29" event={"ID":"09f1d198-7f7a-4cea-b8fd-1806ac828fde","Type":"ContainerStarted","Data":"f7097c7ef0210675395121979c6b19ffd459e4e81009f6a40237f3dab01c8f82"}
Jul 31 14:49:08.221178 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:08.221131    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m8g29" podStartSLOduration=33.504027567 podStartE2EDuration="35.221112629s" totalImagesPullingTime="1.717085062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 14:48:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:08.219873805 +0000 UTC m=+90.931848330" watchObservedRunningTime="2026-07-31 14:49:08.221112629 +0000 UTC m=+90.933087159"
Jul 31 14:49:11.808685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:11.808653    2566 scope.go:117] "RemoveContainer" containerID="3c7923ed42e2b0c7acd540f536d116ce2de320879f7cb9be37d010022a7af8e1"
Jul 31 14:49:12.202021 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.201995    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:49:12.202174 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.202051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" event={"ID":"e00e5e8b-4f02-406d-a8aa-3966ed31acae","Type":"ContainerStarted","Data":"6ad5931168ddb08d8b32171e4e964ddc6ff7720a24f96d2b9acbea7796c07a4a"}
Jul 31 14:49:12.202332 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.202311    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:49:12.220342 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.220295    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68" podStartSLOduration=41.464846647 podStartE2EDuration="45.220276861s" totalImagesPullingTime="3.755430214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:12.219882625 +0000 UTC m=+94.931857154" watchObservedRunningTime="2026-07-31 14:49:12.220276861 +0000 UTC m=+94.932251392"
Jul 31 14:49:12.585720 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.585629    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-n2v68"
Jul 31 14:49:12.767607 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.767575    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-xdnp5"]
Jul 31 14:49:12.769669 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.769646    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:12.772476 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.772427    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8v779\""
Jul 31 14:49:12.772620 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.772572    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 31 14:49:12.772793 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.772766    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 31 14:49:12.778927 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.778901    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-xdnp5"]
Jul 31 14:49:12.843563 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.843495    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw6jz\" (UniqueName: \"kubernetes.io/projected/d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99-kube-api-access-tw6jz\") pod \"downloads-77c974ffcf-xdnp5\" (UID: \"d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99\") " pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:12.944523 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.944497    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw6jz\" (UniqueName: \"kubernetes.io/projected/d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99-kube-api-access-tw6jz\") pod \"downloads-77c974ffcf-xdnp5\" (UID: \"d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99\") " pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:12.954605 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:12.954580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw6jz\" (UniqueName: \"kubernetes.io/projected/d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99-kube-api-access-tw6jz\") pod \"downloads-77c974ffcf-xdnp5\" (UID: \"d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99\") " pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:13.080942 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:13.080915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:13.229863 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:13.229841    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-xdnp5"]
Jul 31 14:49:13.232116 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:13.232092    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd510c2f1_c5ab_4eb5_962f_33d3eaaa1b99.slice/crio-5cf899fd43d2205922c7086e801c7f3983b9ab0b440d4d2da4d4aae54ee8335e WatchSource:0}: Error finding container 5cf899fd43d2205922c7086e801c7f3983b9ab0b440d4d2da4d4aae54ee8335e: Status 404 returned error can't find the container with id 5cf899fd43d2205922c7086e801c7f3983b9ab0b440d4d2da4d4aae54ee8335e
Jul 31 14:49:14.210426 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.210389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-xdnp5" event={"ID":"d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99","Type":"ContainerStarted","Data":"5cf899fd43d2205922c7086e801c7f3983b9ab0b440d4d2da4d4aae54ee8335e"}
Jul 31 14:49:14.412562 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.412528    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"]
Jul 31 14:49:14.414852 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.414826    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.421790 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.421681    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xgt57\""
Jul 31 14:49:14.423451 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.421999    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 31 14:49:14.423451 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.422566    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 31 14:49:14.427800 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.427778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 31 14:49:14.438646 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.438621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"]
Jul 31 14:49:14.461488 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.461409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h8wr\" (UniqueName: \"kubernetes.io/projected/614252f9-ade6-4829-99d5-ca836e7223ad-kube-api-access-4h8wr\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.461625 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.461571    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.461685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.461639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.461685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.461670    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/614252f9-ade6-4829-99d5-ca836e7223ad-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.538718 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.538688    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"]
Jul 31 14:49:14.541229 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.541207    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.544141 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.544120    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 31 14:49:14.544263 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.544170    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 31 14:49:14.544263 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.544244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tw7bq\""
Jul 31 14:49:14.544263 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.544172    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 31 14:49:14.562065 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.562044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.562184 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.562095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.562184 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.562122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/614252f9-ade6-4829-99d5-ca836e7223ad-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.562184 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.562169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h8wr\" (UniqueName: \"kubernetes.io/projected/614252f9-ade6-4829-99d5-ca836e7223ad-kube-api-access-4h8wr\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.563055 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.563028    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/614252f9-ade6-4829-99d5-ca836e7223ad-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.565244 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.565224    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.565325 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.565283    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/614252f9-ade6-4829-99d5-ca836e7223ad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.565382 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.565363    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"]
Jul 31 14:49:14.570133 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.570114    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qlvvq"]
Jul 31 14:49:14.573336 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.572757    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.575930 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.575911    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 31 14:49:14.576267 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.576244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qmq89\""
Jul 31 14:49:14.576385 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.576362    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 31 14:49:14.576459 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.576369    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 31 14:49:14.582610 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.582580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h8wr\" (UniqueName: \"kubernetes.io/projected/614252f9-ade6-4829-99d5-ca836e7223ad-kube-api-access-4h8wr\") pod \"openshift-state-metrics-6bb4d7b4dd-lfqld\" (UID: \"614252f9-ade6-4829-99d5-ca836e7223ad\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.663034 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.662996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43b700ef-dc72-404b-9bb0-dc8da84f512b-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663194 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663194 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-textfile\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663194 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663202    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zvvt\" (UniqueName: \"kubernetes.io/projected/a31b6a53-e665-475a-a54f-6330a32ff800-kube-api-access-4zvvt\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-sys\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-metrics-client-ca\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663369 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g68kk\" (UniqueName: \"kubernetes.io/projected/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-api-access-g68kk\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663564 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663392    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-root\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663564 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663445    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663564 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663564 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.663708 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-tls\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.663708 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.663625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-wtmp\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.730631 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.730549    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"
Jul 31 14:49:14.764137 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43b700ef-dc72-404b-9bb0-dc8da84f512b-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.764272 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764144    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.764272 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-textfile\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764272 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764467 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764467 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zvvt\" (UniqueName: \"kubernetes.io/projected/a31b6a53-e665-475a-a54f-6330a32ff800-kube-api-access-4zvvt\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764467 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-sys\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764467 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-sys\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764661 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-textfile\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764661 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764539    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43b700ef-dc72-404b-9bb0-dc8da84f512b-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.764862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-metrics-client-ca\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g68kk\" (UniqueName: \"kubernetes.io/projected/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-api-access-g68kk\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.764862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-root\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.764862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.765051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.765051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.765051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-tls\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.764974    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-wtmp\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765051 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765030    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-root\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765156    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-wtmp\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31b6a53-e665-475a-a54f-6330a32ff800-metrics-client-ca\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.765375 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:49:14.765310    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 31 14:49:14.765375 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765322    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.765467 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:49:14.765377    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls podName:43b700ef-dc72-404b-9bb0-dc8da84f512b nodeName:}" failed. No retries permitted until 2026-07-31 14:49:15.265359633 +0000 UTC m=+97.977334142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-v8pwh" (UID: "43b700ef-dc72-404b-9bb0-dc8da84f512b") : secret "kube-state-metrics-tls" not found
Jul 31 14:49:14.765685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.765665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43b700ef-dc72-404b-9bb0-dc8da84f512b-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.767924 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.767905    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.768215 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.768190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.768376 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.768348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a31b6a53-e665-475a-a54f-6330a32ff800-node-exporter-tls\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.775238 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.775177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zvvt\" (UniqueName: \"kubernetes.io/projected/a31b6a53-e665-475a-a54f-6330a32ff800-kube-api-access-4zvvt\") pod \"node-exporter-qlvvq\" (UID: \"a31b6a53-e665-475a-a54f-6330a32ff800\") " pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.783437 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.783406    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g68kk\" (UniqueName: \"kubernetes.io/projected/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-api-access-g68kk\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:14.880239 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.880204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld"]
Jul 31 14:49:14.883381 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:14.883352    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614252f9_ade6_4829_99d5_ca836e7223ad.slice/crio-9eff0870a4e5c99a0b2b5b6f5cff2668f26a044539cf457ff2be40cb65255a94 WatchSource:0}: Error finding container 9eff0870a4e5c99a0b2b5b6f5cff2668f26a044539cf457ff2be40cb65255a94: Status 404 returned error can't find the container with id 9eff0870a4e5c99a0b2b5b6f5cff2668f26a044539cf457ff2be40cb65255a94
Jul 31 14:49:14.893926 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:14.893903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlvvq"
Jul 31 14:49:14.905130 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:14.905106    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31b6a53_e665_475a_a54f_6330a32ff800.slice/crio-973c6bb1c44f6775ee9c9681a9a0b78c316465ea06814c720057c608a5ed3912 WatchSource:0}: Error finding container 973c6bb1c44f6775ee9c9681a9a0b78c316465ea06814c720057c608a5ed3912: Status 404 returned error can't find the container with id 973c6bb1c44f6775ee9c9681a9a0b78c316465ea06814c720057c608a5ed3912
Jul 31 14:49:15.216619 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.216570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld" event={"ID":"614252f9-ade6-4829-99d5-ca836e7223ad","Type":"ContainerStarted","Data":"6d6098f3b283e53a5b589abd2c21883abd6ef551e610ba6dd1746370bc341360"}
Jul 31 14:49:15.217083 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.216638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld" event={"ID":"614252f9-ade6-4829-99d5-ca836e7223ad","Type":"ContainerStarted","Data":"e29e73e415f8117d306417c582008712d145c15d90edb80592690af5c514e8d9"}
Jul 31 14:49:15.217083 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.216669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld" event={"ID":"614252f9-ade6-4829-99d5-ca836e7223ad","Type":"ContainerStarted","Data":"9eff0870a4e5c99a0b2b5b6f5cff2668f26a044539cf457ff2be40cb65255a94"}
Jul 31 14:49:15.217959 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.217939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlvvq" event={"ID":"a31b6a53-e665-475a-a54f-6330a32ff800","Type":"ContainerStarted","Data":"973c6bb1c44f6775ee9c9681a9a0b78c316465ea06814c720057c608a5ed3912"}
Jul 31 14:49:15.270862 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.270774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:15.273322 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.273300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43b700ef-dc72-404b-9bb0-dc8da84f512b-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-v8pwh\" (UID: \"43b700ef-dc72-404b-9bb0-dc8da84f512b\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:15.324484 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.324447    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:49:15.327878 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.327849    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.330748 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.330561    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 31 14:49:15.331056 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331030    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 31 14:49:15.331146 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331075    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 31 14:49:15.331146 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331126    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 31 14:49:15.331345 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331322    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 31 14:49:15.331476 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331352    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 31 14:49:15.331476 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331408    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 31 14:49:15.331577 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331544    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 31 14:49:15.331621 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331577    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 31 14:49:15.331621 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.331615    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qzhtj\""
Jul 31 14:49:15.362029 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.362000    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:49:15.372116 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372241 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb7np\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372581 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372453    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372581 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.372581 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.372564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.451500 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.451475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"
Jul 31 14:49:15.473410 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473380    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473567 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473567 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473467    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473567 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473567 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473660    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb7np\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.473770 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.473762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.474536 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.474512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.474609 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.474567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.475711 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.475680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.478395 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.477989    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.478395 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.477999    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.478395 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.478358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.479683 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.479659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.480879 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.480827    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.481202 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.481176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.481288 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.481241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.481288 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.481253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.481397 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.481318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.496299 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.496182    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb7np\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np\") pod \"alertmanager-main-0\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.638823 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.638581    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:49:15.643337 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.642940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh"]
Jul 31 14:49:15.654077 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:15.654047    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b700ef_dc72_404b_9bb0_dc8da84f512b.slice/crio-5abd285a954111da2c73c3916299e539863a0a573b2547a557c4ac3c8472e26b WatchSource:0}: Error finding container 5abd285a954111da2c73c3916299e539863a0a573b2547a557c4ac3c8472e26b: Status 404 returned error can't find the container with id 5abd285a954111da2c73c3916299e539863a0a573b2547a557c4ac3c8472e26b
Jul 31 14:49:15.853163 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:15.853069    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:49:15.856660 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:15.856620    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f705307_7a9e_4d93_a1d4_d0985c0a3ad5.slice/crio-1db7c6437911557673ab434100dcb01850fca9f00d745fdaaef2a6205a85a442 WatchSource:0}: Error finding container 1db7c6437911557673ab434100dcb01850fca9f00d745fdaaef2a6205a85a442: Status 404 returned error can't find the container with id 1db7c6437911557673ab434100dcb01850fca9f00d745fdaaef2a6205a85a442
Jul 31 14:49:16.080899 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.080846    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:16.083876 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.083847    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"image-registry-6f4655fff-5zqmc\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") " pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:16.182240 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.182206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:49:16.185119 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.185065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/baca90d9-9cb5-43f1-a5b5-47826d395d31-metrics-tls\") pod \"dns-default-n2xrq\" (UID: \"baca90d9-9cb5-43f1-a5b5-47826d395d31\") " pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:49:16.224409 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.224360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"1db7c6437911557673ab434100dcb01850fca9f00d745fdaaef2a6205a85a442"}
Jul 31 14:49:16.225761 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.225734    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh" event={"ID":"43b700ef-dc72-404b-9bb0-dc8da84f512b","Type":"ContainerStarted","Data":"5abd285a954111da2c73c3916299e539863a0a573b2547a557c4ac3c8472e26b"}
Jul 31 14:49:16.352430 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.352396    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4fm2\""
Jul 31 14:49:16.358669 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.358643    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:16.378746 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.378719    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-46zx7\""
Jul 31 14:49:16.385126 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.385100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:49:16.822225 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.822143    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2xrq"]
Jul 31 14:49:16.825692 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:16.825477    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:49:17.235162 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.234438    2566 generic.go:356] "Generic (PLEG): container finished" podID="a31b6a53-e665-475a-a54f-6330a32ff800" containerID="0bfb051ad8e4cba4cc4264d2d1a2526680a7ed62165d7b3d019817f8fd2e3200" exitCode=0
Jul 31 14:49:17.235162 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.234545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlvvq" event={"ID":"a31b6a53-e665-475a-a54f-6330a32ff800","Type":"ContainerDied","Data":"0bfb051ad8e4cba4cc4264d2d1a2526680a7ed62165d7b3d019817f8fd2e3200"}
Jul 31 14:49:17.238253 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.238221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld" event={"ID":"614252f9-ade6-4829-99d5-ca836e7223ad","Type":"ContainerStarted","Data":"c0b02c14a69594a57427a00003d771057ce4eec43aae3ef77773fe8344934ad6"}
Jul 31 14:49:17.240324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.240255    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2xrq" event={"ID":"baca90d9-9cb5-43f1-a5b5-47826d395d31","Type":"ContainerStarted","Data":"57ef31b456fda374c503f2b3cb4f19bd6211c653bf8f135012ced605841e0bde"}
Jul 31 14:49:17.242187 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.242082    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" event={"ID":"70c5dbd2-f468-4343-976d-0a117b21bd72","Type":"ContainerStarted","Data":"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"}
Jul 31 14:49:17.242187 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.242120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" event={"ID":"70c5dbd2-f468-4343-976d-0a117b21bd72","Type":"ContainerStarted","Data":"47dc023f7ca0de1a8ad078c5c386d5a14e7cf84d81f1ff849b89eb4516844ec3"}
Jul 31 14:49:17.242187 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.242144    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:17.302375 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.302325    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-lfqld" podStartSLOduration=1.882300264 podStartE2EDuration="3.302309105s" totalImagesPullingTime="1.420008841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:17.300879101 +0000 UTC m=+100.012853632" watchObservedRunningTime="2026-07-31 14:49:17.302309105 +0000 UTC m=+100.014283627"
Jul 31 14:49:17.343411 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:17.343361    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" podStartSLOduration=98.343342708 podStartE2EDuration="1m38.343342708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:17.342521913 +0000 UTC m=+100.054496468" watchObservedRunningTime="2026-07-31 14:49:17.343342708 +0000 UTC m=+100.055317237"
Jul 31 14:49:17.928723 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:49:17.928457    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f705307_7a9e_4d93_a1d4_d0985c0a3ad5.slice/crio-6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de.scope\": RecentStats: unable to find data in memory cache]"
Jul 31 14:49:18.251943 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.251905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh" event={"ID":"43b700ef-dc72-404b-9bb0-dc8da84f512b","Type":"ContainerStarted","Data":"bc2b03f6cb645a3df887e033e3d9380558d684bcf0ba96cb4f18507799f1acfe"}
Jul 31 14:49:18.252387 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.252071    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh" event={"ID":"43b700ef-dc72-404b-9bb0-dc8da84f512b","Type":"ContainerStarted","Data":"8099dfb92703c98a4e7ce7b34912d516b8669c670536ae2b6dbcddd6cfcf76f0"}
Jul 31 14:49:18.252387 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.252112    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh" event={"ID":"43b700ef-dc72-404b-9bb0-dc8da84f512b","Type":"ContainerStarted","Data":"d41d866a8c86f9c94748045246b7d3347c58bbafdfa298012a33ba3d26a3667f"}
Jul 31 14:49:18.254489 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.254460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlvvq" event={"ID":"a31b6a53-e665-475a-a54f-6330a32ff800","Type":"ContainerStarted","Data":"b1e2acc6e7ebc51ddd4d8d090a8d6968550bad7ff2fc019176bc7b73ddb4ed8f"}
Jul 31 14:49:18.254620 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.254517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlvvq" event={"ID":"a31b6a53-e665-475a-a54f-6330a32ff800","Type":"ContainerStarted","Data":"aad017ec20c1aa99c9ad985cd8defc413a21ac16b6ff6e51ba5be5c3c612ef9a"}
Jul 31 14:49:18.256688 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.256664    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de" exitCode=0
Jul 31 14:49:18.256906 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.256881    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"}
Jul 31 14:49:18.278176 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.278075    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-v8pwh" podStartSLOduration=2.169345601 podStartE2EDuration="4.278053895s" totalImagesPullingTime="2.108708294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:18.277477113 +0000 UTC m=+100.989451643" watchObservedRunningTime="2026-07-31 14:49:18.278053895 +0000 UTC m=+100.990028425"
Jul 31 14:49:18.353759 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:18.353703    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qlvvq" podStartSLOduration=2.716362432 podStartE2EDuration="4.353686852s" totalImagesPullingTime="1.526808464s" totalInitContainerRuntime="110.515956ms" isStatefulPod=true podCreationTimestamp="2026-07-31 14:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:18.352530384 +0000 UTC m=+101.064504913" watchObservedRunningTime="2026-07-31 14:49:18.353686852 +0000 UTC m=+101.065661380"
Jul 31 14:49:19.005326 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.005292    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"]
Jul 31 14:49:19.009778 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.009739    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.012566 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.012540    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8rcpk\""
Jul 31 14:49:19.013058 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.012871    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 31 14:49:19.031014 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.030893    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"]
Jul 31 14:49:19.117927 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.117893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7936908-70fc-48fb-8e2e-ae9828f58b79-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.118101 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.117957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxhf\" (UniqueName: \"kubernetes.io/projected/f7936908-70fc-48fb-8e2e-ae9828f58b79-kube-api-access-8nxhf\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.219176 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.219145    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7936908-70fc-48fb-8e2e-ae9828f58b79-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.219352 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.219198    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxhf\" (UniqueName: \"kubernetes.io/projected/f7936908-70fc-48fb-8e2e-ae9828f58b79-kube-api-access-8nxhf\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.222185 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.222156    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7936908-70fc-48fb-8e2e-ae9828f58b79-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.230414 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.230388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxhf\" (UniqueName: \"kubernetes.io/projected/f7936908-70fc-48fb-8e2e-ae9828f58b79-kube-api-access-8nxhf\") pod \"monitoring-plugin-6dffbbd67-m256w\" (UID: \"f7936908-70fc-48fb-8e2e-ae9828f58b79\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.264026 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.263938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2xrq" event={"ID":"baca90d9-9cb5-43f1-a5b5-47826d395d31","Type":"ContainerStarted","Data":"ee5fdf6d038f21ecc9f8f6460d40f47a2bf63c8235adea7dad7f395394d67148"}
Jul 31 14:49:19.264026 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.263998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2xrq" event={"ID":"baca90d9-9cb5-43f1-a5b5-47826d395d31","Type":"ContainerStarted","Data":"d1c72e1d0283ba9d87ec7112a091290906d999876218c425a8b429c857339d8b"}
Jul 31 14:49:19.264721 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.264670    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:49:19.299590 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.299488    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n2xrq" podStartSLOduration=65.253373287 podStartE2EDuration="1m7.299467448s" totalImagesPullingTime="2.046094161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:19.299248306 +0000 UTC m=+102.011222881" watchObservedRunningTime="2026-07-31 14:49:19.299467448 +0000 UTC m=+102.011441977"
Jul 31 14:49:19.323677 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.323655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:19.481452 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:19.481365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"]
Jul 31 14:49:19.484586 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:19.484548    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7936908_70fc_48fb_8e2e_ae9828f58b79.slice/crio-79f2f0be6fd80a800f4790b4e3dec5e84c66fef3b79b553da877e38aec1776ce WatchSource:0}: Error finding container 79f2f0be6fd80a800f4790b4e3dec5e84c66fef3b79b553da877e38aec1776ce: Status 404 returned error can't find the container with id 79f2f0be6fd80a800f4790b4e3dec5e84c66fef3b79b553da877e38aec1776ce
Jul 31 14:49:20.115364 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:20.115250    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7vzl5"
Jul 31 14:49:20.269319 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:20.269282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w" event={"ID":"f7936908-70fc-48fb-8e2e-ae9828f58b79","Type":"ContainerStarted","Data":"79f2f0be6fd80a800f4790b4e3dec5e84c66fef3b79b553da877e38aec1776ce"}
Jul 31 14:49:20.271420 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:20.271335    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"}
Jul 31 14:49:20.642613 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:20.642585    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:49:21.281969 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:21.281932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"}
Jul 31 14:49:21.282331 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:21.282012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"}
Jul 31 14:49:21.282331 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:21.282053    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"}
Jul 31 14:49:21.282331 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:21.282091    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"}
Jul 31 14:49:23.294029 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.293994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerStarted","Data":"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"}
Jul 31 14:49:23.295589 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.295561    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w" event={"ID":"f7936908-70fc-48fb-8e2e-ae9828f58b79","Type":"ContainerStarted","Data":"fd7245ded953c3f613567ce60cd15a824a0572efde44a96fe58439df690ce53c"}
Jul 31 14:49:23.295854 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.295834    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:23.301936 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.301915    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w"
Jul 31 14:49:23.329074 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.329028    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.955411749 podStartE2EDuration="8.329013106s" totalImagesPullingTime="5.348235586s" totalInitContainerRuntime="25.365771ms" isStatefulPod=false podCreationTimestamp="2026-07-31 14:49:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:23.328027611 +0000 UTC m=+106.040002142" watchObservedRunningTime="2026-07-31 14:49:23.329013106 +0000 UTC m=+106.040987635"
Jul 31 14:49:23.351497 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:23.351436    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-m256w" podStartSLOduration=2.5480314330000002 podStartE2EDuration="5.35141323s" totalImagesPullingTime="2.803381797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:23.349171354 +0000 UTC m=+106.061145883" watchObservedRunningTime="2026-07-31 14:49:23.35141323 +0000 UTC m=+106.063387760"
Jul 31 14:49:30.284917 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:30.284882    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n2xrq"
Jul 31 14:49:31.335311 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:31.335271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-xdnp5" event={"ID":"d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99","Type":"ContainerStarted","Data":"8049ad28bbfff7f3543812f4004a1f065df06e686508e2eed73f63e2d0917670"}
Jul 31 14:49:31.335775 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:31.335457    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:31.350574 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:31.350545    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-xdnp5"
Jul 31 14:49:31.364089 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:31.364032    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-xdnp5" podStartSLOduration=1.899962165 podStartE2EDuration="19.364012109s" totalImagesPullingTime="17.464049944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:31.363362669 +0000 UTC m=+114.075337198" watchObservedRunningTime="2026-07-31 14:49:31.364012109 +0000 UTC m=+114.075986640"
Jul 31 14:49:38.366540 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:38.366502    2566 generic.go:356] "Generic (PLEG): container finished" podID="a6802a36-92bb-4e6a-a8ad-d86898381e2c" containerID="908f0b4e79a12334b60b0d5f5de8b445393caa995e0f32861a99048e073c767b" exitCode=0
Jul 31 14:49:38.367057 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:38.366582    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-lwjqb" event={"ID":"a6802a36-92bb-4e6a-a8ad-d86898381e2c","Type":"ContainerDied","Data":"908f0b4e79a12334b60b0d5f5de8b445393caa995e0f32861a99048e073c767b"}
Jul 31 14:49:38.367127 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:38.367055    2566 scope.go:117] "RemoveContainer" containerID="908f0b4e79a12334b60b0d5f5de8b445393caa995e0f32861a99048e073c767b"
Jul 31 14:49:39.180040 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:39.180009    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-587cb6667d-857jd_2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179/router/0.log"
Jul 31 14:49:39.200274 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:39.200248    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5qq4_dbd836fa-93a0-4c34-a332-c3606ef36615/serve-healthcheck-canary/0.log"
Jul 31 14:49:39.373192 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:39.373153    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-lwjqb" event={"ID":"a6802a36-92bb-4e6a-a8ad-d86898381e2c","Type":"ContainerStarted","Data":"3642e17e8ba9294724ed05f9e3b01e629101284794e67f99be765a5c15465f6f"}
Jul 31 14:49:40.648767 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:40.648738    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:45.664308 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.664254    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" podUID="70c5dbd2-f468-4343-976d-0a117b21bd72" containerName="registry" containerID="cri-o://289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356" gracePeriod=30
Jul 31 14:49:45.943407 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.943384    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:45.975204 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975179    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975234    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975324 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975271    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7lsg\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975444 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975357    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975444 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975397    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975444 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975422    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975455    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975593 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975523    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca\") pod \"70c5dbd2-f468-4343-976d-0a117b21bd72\" (UID: \"70c5dbd2-f468-4343-976d-0a117b21bd72\") "
Jul 31 14:49:45.975801 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.975769    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 14:49:45.976213 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.976188    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 14:49:45.977781 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.977752    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:49:45.977901 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.977798    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg" (OuterVolumeSpecName: "kube-api-access-x7lsg") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "kube-api-access-x7lsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:49:45.978165 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.978131    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:49:45.978248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.978200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:49:45.978419 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.978393    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:49:45.984126 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:45.984099    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "70c5dbd2-f468-4343-976d-0a117b21bd72" (UID: "70c5dbd2-f468-4343-976d-0a117b21bd72"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 14:49:46.076882 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076856    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-bound-sa-token\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076879    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70c5dbd2-f468-4343-976d-0a117b21bd72-ca-trust-extracted\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076901    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7lsg\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-kube-api-access-x7lsg\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076917    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-certificates\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076931    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-installation-pull-secrets\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076943    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70c5dbd2-f468-4343-976d-0a117b21bd72-registry-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076957    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70c5dbd2-f468-4343-976d-0a117b21bd72-image-registry-private-configuration\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.076999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.076970    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70c5dbd2-f468-4343-976d-0a117b21bd72-trusted-ca\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:49:46.400061 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.400030    2566 generic.go:356] "Generic (PLEG): container finished" podID="70c5dbd2-f468-4343-976d-0a117b21bd72" containerID="289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356" exitCode=0
Jul 31 14:49:46.400308 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.400084    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc"
Jul 31 14:49:46.400308 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.400120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" event={"ID":"70c5dbd2-f468-4343-976d-0a117b21bd72","Type":"ContainerDied","Data":"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"}
Jul 31 14:49:46.400308 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.400173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f4655fff-5zqmc" event={"ID":"70c5dbd2-f468-4343-976d-0a117b21bd72","Type":"ContainerDied","Data":"47dc023f7ca0de1a8ad078c5c386d5a14e7cf84d81f1ff849b89eb4516844ec3"}
Jul 31 14:49:46.400308 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.400196    2566 scope.go:117] "RemoveContainer" containerID="289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"
Jul 31 14:49:46.410957 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.410934    2566 scope.go:117] "RemoveContainer" containerID="289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"
Jul 31 14:49:46.411343 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:49:46.411214    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356\": container with ID starting with 289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356 not found: ID does not exist" containerID="289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"
Jul 31 14:49:46.411343 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.411244    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356"} err="failed to get container status \"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356\": rpc error: code = NotFound desc = could not find container \"289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356\": container with ID starting with 289bf4ed9ac91a53d0296eba988555868b90fd82b8447bbfd13acff512714356 not found: ID does not exist"
Jul 31 14:49:46.428827 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.428782    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:49:46.438132 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:46.438108    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f4655fff-5zqmc"]
Jul 31 14:49:47.589733 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.589698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:49:47.591994 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.591973    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29e6e9-f8e0-451e-b658-cbf0ed5ab089-metrics-certs\") pod \"network-metrics-daemon-n8x7h\" (UID: \"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089\") " pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:49:47.636053 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.636030    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4v8n\""
Jul 31 14:49:47.642685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.642666    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8x7h"
Jul 31 14:49:47.800035 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.800013    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n8x7h"]
Jul 31 14:49:47.802240 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:49:47.802216    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b29e6e9_f8e0_451e_b658_cbf0ed5ab089.slice/crio-322343c87072195f727f6a8006e536289bafafea2a0d8b873fa18021ce8b0354 WatchSource:0}: Error finding container 322343c87072195f727f6a8006e536289bafafea2a0d8b873fa18021ce8b0354: Status 404 returned error can't find the container with id 322343c87072195f727f6a8006e536289bafafea2a0d8b873fa18021ce8b0354
Jul 31 14:49:47.814783 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:47.814757    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c5dbd2-f468-4343-976d-0a117b21bd72" path="/var/lib/kubelet/pods/70c5dbd2-f468-4343-976d-0a117b21bd72/volumes"
Jul 31 14:49:48.409921 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:48.409887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8x7h" event={"ID":"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089","Type":"ContainerStarted","Data":"322343c87072195f727f6a8006e536289bafafea2a0d8b873fa18021ce8b0354"}
Jul 31 14:49:50.421345 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:50.421309    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8x7h" event={"ID":"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089","Type":"ContainerStarted","Data":"a2b0cb13a55ed440038fd27280363e353fda0c1deba059125207a8d2ad7e574e"}
Jul 31 14:49:50.421717 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:50.421369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8x7h" event={"ID":"1b29e6e9-f8e0-451e-b658-cbf0ed5ab089","Type":"ContainerStarted","Data":"f0824458d77c1cb53e71622b6cd30b1b38237193053bc539f02d28b2b77d1cbc"}
Jul 31 14:49:50.450665 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:50.450610    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n8x7h" podStartSLOduration=130.909212487 podStartE2EDuration="2m12.450593479s" totalImagesPullingTime="1.541380992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:49:50.44910458 +0000 UTC m=+133.161079109" watchObservedRunningTime="2026-07-31 14:49:50.450593479 +0000 UTC m=+133.162568007"
Jul 31 14:49:59.461161 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:59.461127    2566 generic.go:356] "Generic (PLEG): container finished" podID="60deba42-8140-4e36-bfee-4b289d360099" containerID="8a354aee6feb3253c6d197205187e55cd129267fd61db29ab579163538cc5e9a" exitCode=0
Jul 31 14:49:59.461584 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:59.461206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j" event={"ID":"60deba42-8140-4e36-bfee-4b289d360099","Type":"ContainerDied","Data":"8a354aee6feb3253c6d197205187e55cd129267fd61db29ab579163538cc5e9a"}
Jul 31 14:49:59.461584 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:49:59.461560    2566 scope.go:117] "RemoveContainer" containerID="8a354aee6feb3253c6d197205187e55cd129267fd61db29ab579163538cc5e9a"
Jul 31 14:50:00.467925 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:00.467895    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5tc6j" event={"ID":"60deba42-8140-4e36-bfee-4b289d360099","Type":"ContainerStarted","Data":"60d0e9a664b994c4c8adb984e8af348ec7a111f1e6934fc4307d9359c641e4c3"}
Jul 31 14:50:08.503282 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:08.503247    2566 generic.go:356] "Generic (PLEG): container finished" podID="b3aad374-423d-4a82-8a0d-e2947ecc6168" containerID="af06978eab882b1167f09da03f8a6b31b2763545c065275deb23ba57ff9bde19" exitCode=0
Jul 31 14:50:08.503625 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:08.503318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh" event={"ID":"b3aad374-423d-4a82-8a0d-e2947ecc6168","Type":"ContainerDied","Data":"af06978eab882b1167f09da03f8a6b31b2763545c065275deb23ba57ff9bde19"}
Jul 31 14:50:08.503625 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:08.503595    2566 scope.go:117] "RemoveContainer" containerID="af06978eab882b1167f09da03f8a6b31b2763545c065275deb23ba57ff9bde19"
Jul 31 14:50:09.509782 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:09.509745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-bxpvh" event={"ID":"b3aad374-423d-4a82-8a0d-e2947ecc6168","Type":"ContainerStarted","Data":"883095f6cedd05aed21f5e65dd3074853ea2a7b01a7b4fa2f62001a7b65810b1"}
Jul 31 14:50:34.841671 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841596    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841884    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy" containerID="cri-o://61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5" gracePeriod=120
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841920    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-web" containerID="cri-o://a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f" gracePeriod=120
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841948    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="prom-label-proxy" containerID="cri-o://5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35" gracePeriod=120
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841948    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-metric" containerID="cri-o://220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18" gracePeriod=120
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841869    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="alertmanager" containerID="cri-o://879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2" gracePeriod=120
Jul 31 14:50:34.842167 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:34.841938    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="config-reloader" containerID="cri-o://cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210" gracePeriod=120
Jul 31 14:50:35.612028 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.611993    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35" exitCode=0
Jul 31 14:50:35.612028 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612017    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5" exitCode=0
Jul 31 14:50:35.612028 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612023    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210" exitCode=0
Jul 31 14:50:35.612028 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612030    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2" exitCode=0
Jul 31 14:50:35.612278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"}
Jul 31 14:50:35.612278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612141    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"}
Jul 31 14:50:35.612278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"}
Jul 31 14:50:35.612278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:35.612192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"}
Jul 31 14:50:36.075670 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.075650    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.257466 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257437    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb7np\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257477    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257507    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257523    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257561    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257595    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257618    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.257650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257649    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258054 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258054 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257706    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258054 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257730    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258054 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257769    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258054 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.257798    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db\") pod \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\" (UID: \"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5\") "
Jul 31 14:50:36.258584 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.258510    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 14:50:36.258937 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.258911    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 14:50:36.259030 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.258942    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 14:50:36.260264 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.260233    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.260985 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.260948    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume" (OuterVolumeSpecName: "config-volume") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.261242 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.261139    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.261242 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.261187    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.261242 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.261218    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np" (OuterVolumeSpecName: "kube-api-access-wb7np") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "kube-api-access-wb7np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:50:36.261383 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.261287    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.261585 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.261568    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out" (OuterVolumeSpecName: "config-out") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 14:50:36.262108 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.262088    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:50:36.264995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.264975    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.271143 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.271121    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config" (OuterVolumeSpecName: "web-config") pod "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" (UID: "9f705307-7a9e-4d93-a1d4-d0985c0a3ad5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:50:36.359250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359215    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-cluster-tls-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359250    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-metrics-client-ca\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359268    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359284    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-main-db\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359298    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb7np\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-kube-api-access-wb7np\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359314    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-web-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359327    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-volume\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359340    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359355    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359367    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-secret-alertmanager-main-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359379    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-tls-assets\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359393    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.359413 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.359405    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5-config-out\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:50:36.618832 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618732    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18" exitCode=0
Jul 31 14:50:36.618832 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618759    2566 generic.go:356] "Generic (PLEG): container finished" podID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerID="a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f" exitCode=0
Jul 31 14:50:36.618832 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618822    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.619072 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618838    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"}
Jul 31 14:50:36.619072 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"}
Jul 31 14:50:36.619072 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618933    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f705307-7a9e-4d93-a1d4-d0985c0a3ad5","Type":"ContainerDied","Data":"1db7c6437911557673ab434100dcb01850fca9f00d745fdaaef2a6205a85a442"}
Jul 31 14:50:36.619072 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.618970    2566 scope.go:117] "RemoveContainer" containerID="5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"
Jul 31 14:50:36.627388 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.627369    2566 scope.go:117] "RemoveContainer" containerID="220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"
Jul 31 14:50:36.634346 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.634329    2566 scope.go:117] "RemoveContainer" containerID="61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"
Jul 31 14:50:36.640685 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.640667    2566 scope.go:117] "RemoveContainer" containerID="a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"
Jul 31 14:50:36.648027 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.648010    2566 scope.go:117] "RemoveContainer" containerID="cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"
Jul 31 14:50:36.648184 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.648163    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:36.654650 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.654631    2566 scope.go:117] "RemoveContainer" containerID="879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"
Jul 31 14:50:36.659738 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.656089    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:36.665738 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.665718    2566 scope.go:117] "RemoveContainer" containerID="6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"
Jul 31 14:50:36.672040 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672023    2566 scope.go:117] "RemoveContainer" containerID="5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"
Jul 31 14:50:36.672269 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.672250    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35\": container with ID starting with 5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35 not found: ID does not exist" containerID="5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"
Jul 31 14:50:36.672322 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672276    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"} err="failed to get container status \"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35\": rpc error: code = NotFound desc = could not find container \"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35\": container with ID starting with 5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35 not found: ID does not exist"
Jul 31 14:50:36.672322 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672292    2566 scope.go:117] "RemoveContainer" containerID="220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"
Jul 31 14:50:36.672532 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.672514    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18\": container with ID starting with 220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18 not found: ID does not exist" containerID="220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"
Jul 31 14:50:36.672579 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672537    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"} err="failed to get container status \"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18\": rpc error: code = NotFound desc = could not find container \"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18\": container with ID starting with 220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18 not found: ID does not exist"
Jul 31 14:50:36.672579 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672554    2566 scope.go:117] "RemoveContainer" containerID="61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"
Jul 31 14:50:36.672782 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.672765    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5\": container with ID starting with 61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5 not found: ID does not exist" containerID="61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"
Jul 31 14:50:36.672856 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672787    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"} err="failed to get container status \"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5\": rpc error: code = NotFound desc = could not find container \"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5\": container with ID starting with 61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5 not found: ID does not exist"
Jul 31 14:50:36.672856 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.672826    2566 scope.go:117] "RemoveContainer" containerID="a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"
Jul 31 14:50:36.673049 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.673035    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f\": container with ID starting with a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f not found: ID does not exist" containerID="a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"
Jul 31 14:50:36.673090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673051    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"} err="failed to get container status \"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f\": rpc error: code = NotFound desc = could not find container \"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f\": container with ID starting with a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f not found: ID does not exist"
Jul 31 14:50:36.673090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673063    2566 scope.go:117] "RemoveContainer" containerID="cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"
Jul 31 14:50:36.673269 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.673253    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210\": container with ID starting with cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210 not found: ID does not exist" containerID="cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"
Jul 31 14:50:36.673309 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673280    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"} err="failed to get container status \"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210\": rpc error: code = NotFound desc = could not find container \"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210\": container with ID starting with cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210 not found: ID does not exist"
Jul 31 14:50:36.673309 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673294    2566 scope.go:117] "RemoveContainer" containerID="879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"
Jul 31 14:50:36.673504 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.673481    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2\": container with ID starting with 879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2 not found: ID does not exist" containerID="879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"
Jul 31 14:50:36.673542 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673508    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"} err="failed to get container status \"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2\": rpc error: code = NotFound desc = could not find container \"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2\": container with ID starting with 879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2 not found: ID does not exist"
Jul 31 14:50:36.673542 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673522    2566 scope.go:117] "RemoveContainer" containerID="6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"
Jul 31 14:50:36.673732 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:50:36.673717    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de\": container with ID starting with 6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de not found: ID does not exist" containerID="6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"
Jul 31 14:50:36.673779 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673734    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"} err="failed to get container status \"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de\": rpc error: code = NotFound desc = could not find container \"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de\": container with ID starting with 6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de not found: ID does not exist"
Jul 31 14:50:36.673779 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673747    2566 scope.go:117] "RemoveContainer" containerID="5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"
Jul 31 14:50:36.673995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673978    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35"} err="failed to get container status \"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35\": rpc error: code = NotFound desc = could not find container \"5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35\": container with ID starting with 5ab43d8d17bff2405eaa87b25c4bef8b3ef71221061b0c7a15908d5a8394cb35 not found: ID does not exist"
Jul 31 14:50:36.674031 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.673996    2566 scope.go:117] "RemoveContainer" containerID="220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"
Jul 31 14:50:36.674201 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674183    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18"} err="failed to get container status \"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18\": rpc error: code = NotFound desc = could not find container \"220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18\": container with ID starting with 220a3daacb20c70d91cb1b22e2bc809b3dde645c57a22b65e91a242894917a18 not found: ID does not exist"
Jul 31 14:50:36.674255 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674200    2566 scope.go:117] "RemoveContainer" containerID="61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"
Jul 31 14:50:36.674393 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674377    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5"} err="failed to get container status \"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5\": rpc error: code = NotFound desc = could not find container \"61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5\": container with ID starting with 61c57a3823b0dc44e784b0eb483e0eb5e947d65feca75bb53128943f76ea94b5 not found: ID does not exist"
Jul 31 14:50:36.674433 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674393    2566 scope.go:117] "RemoveContainer" containerID="a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"
Jul 31 14:50:36.674566 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674552    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f"} err="failed to get container status \"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f\": rpc error: code = NotFound desc = could not find container \"a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f\": container with ID starting with a515a1fae536a8ca81d2d5d404e94773de512b2a2a197c5921714d2a573bbd2f not found: ID does not exist"
Jul 31 14:50:36.674566 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674565    2566 scope.go:117] "RemoveContainer" containerID="cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"
Jul 31 14:50:36.674748 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674735    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210"} err="failed to get container status \"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210\": rpc error: code = NotFound desc = could not find container \"cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210\": container with ID starting with cc7fc5bf7e5842949d72e9be18da3203f989e53bec91e49429d540db9acd1210 not found: ID does not exist"
Jul 31 14:50:36.674748 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674747    2566 scope.go:117] "RemoveContainer" containerID="879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"
Jul 31 14:50:36.674928 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674915    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2"} err="failed to get container status \"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2\": rpc error: code = NotFound desc = could not find container \"879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2\": container with ID starting with 879591e1052c91932082d580347617f7db7f76296b53138187277836d8012ac2 not found: ID does not exist"
Jul 31 14:50:36.674928 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.674928    2566 scope.go:117] "RemoveContainer" containerID="6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"
Jul 31 14:50:36.675143 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.675126    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de"} err="failed to get container status \"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de\": rpc error: code = NotFound desc = could not find container \"6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de\": container with ID starting with 6b1548695bd9969519ddacbca367550bddb9151adb6750f13508d5ad7e16d1de not found: ID does not exist"
Jul 31 14:50:36.689675 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.689655    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:36.690099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690080    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690101    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690110    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="init-config-reloader"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690116    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="init-config-reloader"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690129    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-web"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690135    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-web"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690141    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c5dbd2-f468-4343-976d-0a117b21bd72" containerName="registry"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690146    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c5dbd2-f468-4343-976d-0a117b21bd72" containerName="registry"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690155    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="config-reloader"
Jul 31 14:50:36.690158 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690160    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="config-reloader"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690166    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-metric"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690171    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-metric"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690183    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="alertmanager"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690188    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="alertmanager"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690193    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="prom-label-proxy"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690198    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="prom-label-proxy"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690243    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-metric"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690250    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="alertmanager"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690257    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c5dbd2-f468-4343-976d-0a117b21bd72" containerName="registry"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690263    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy-web"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690270    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="prom-label-proxy"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690276    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="config-reloader"
Jul 31 14:50:36.690415 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.690281    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" containerName="kube-rbac-proxy"
Jul 31 14:50:36.693541 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.693526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.696556 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.696538    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 31 14:50:36.697089 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697072    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 31 14:50:36.697319 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697300    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 31 14:50:36.697551 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 31 14:50:36.697551 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697481    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 31 14:50:36.697701 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697570    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qzhtj\""
Jul 31 14:50:36.697701 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697567    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 31 14:50:36.697929 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.697916    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 31 14:50:36.698200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.698183    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 31 14:50:36.704448 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.704428    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 31 14:50:36.710254 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.710235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:36.862924 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.862895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863063 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.862931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s97jl\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-kube-api-access-s97jl\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863063 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.862956    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863063 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863162    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-out\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863313    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863334    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.863412 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.863368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-web-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963684 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-web-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s97jl\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-kube-api-access-s97jl\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.963849 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963831    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-out\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963955    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.963981    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.964008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964152 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.964032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964440 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.964361    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.964772 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.964744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.965206 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.965177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aae37fa-d9cb-426f-a6f8-c328ca269408-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.966603 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.966577    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.966698 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.966585    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-web-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.966781 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.966760    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.966925 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.966899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.967178 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.967154    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.967278 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.967262    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.967339 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.967299    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.967517 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.967495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aae37fa-d9cb-426f-a6f8-c328ca269408-config-out\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.968457 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.968439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aae37fa-d9cb-426f-a6f8-c328ca269408-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:36.972408 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:36.972389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s97jl\" (UniqueName: \"kubernetes.io/projected/0aae37fa-d9cb-426f-a6f8-c328ca269408-kube-api-access-s97jl\") pod \"alertmanager-main-0\" (UID: \"0aae37fa-d9cb-426f-a6f8-c328ca269408\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:37.001908 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.001887    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 31 14:50:37.137092 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.137065    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 31 14:50:37.138997 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:50:37.138968    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aae37fa_d9cb_426f_a6f8_c328ca269408.slice/crio-7196ad97ba46c90b833764e14ca70d127762ab0c4a5a4d96bafb17f081cee18e WatchSource:0}: Error finding container 7196ad97ba46c90b833764e14ca70d127762ab0c4a5a4d96bafb17f081cee18e: Status 404 returned error can't find the container with id 7196ad97ba46c90b833764e14ca70d127762ab0c4a5a4d96bafb17f081cee18e
Jul 31 14:50:37.624995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.624907    2566 generic.go:356] "Generic (PLEG): container finished" podID="0aae37fa-d9cb-426f-a6f8-c328ca269408" containerID="e1ea5541a7b1281ca9e8fc71faaa07cba482323c357e1fe137bdd957b090a3c7" exitCode=0
Jul 31 14:50:37.624995 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.624945    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerDied","Data":"e1ea5541a7b1281ca9e8fc71faaa07cba482323c357e1fe137bdd957b090a3c7"}
Jul 31 14:50:37.625214 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.625005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"7196ad97ba46c90b833764e14ca70d127762ab0c4a5a4d96bafb17f081cee18e"}
Jul 31 14:50:37.812764 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:37.812738    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f705307-7a9e-4d93-a1d4-d0985c0a3ad5" path="/var/lib/kubelet/pods/9f705307-7a9e-4d93-a1d4-d0985c0a3ad5/volumes"
Jul 31 14:50:38.632290 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632258    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"a7b3926e437298e52eb05b0dab4d67cec337bc6a769d6fef86680647478a9a45"}
Jul 31 14:50:38.632628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"80fd856410e25f94983c238d38d59f8a5a3f40b70875d5aba418c5a0ac49c414"}
Jul 31 14:50:38.632628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"de794272af9fe3c2af85c0f46d31891f81125727f04790d7317fad16a10c433d"}
Jul 31 14:50:38.632628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"7b4b2bc7953153b43cade422115e5029dbf7470847e06d75ace9596ae9cce44f"}
Jul 31 14:50:38.632628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"ad10fa3d8f4834728c7d782f7c84b4bcf54dff3a796ee654af4ed0097853f37d"}
Jul 31 14:50:38.632628 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.632424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aae37fa-d9cb-426f-a6f8-c328ca269408","Type":"ContainerStarted","Data":"fd5149543e428b2e378c2639bb7a406b5688284bf90db6e8a80650afa4b2d670"}
Jul 31 14:50:38.763311 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.763263    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.745041102 podStartE2EDuration="2.763247707s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.206605ms" isStatefulPod=false podCreationTimestamp="2026-07-31 14:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:50:38.760949825 +0000 UTC m=+181.472924371" watchObservedRunningTime="2026-07-31 14:50:38.763247707 +0000 UTC m=+181.475222254"
Jul 31 14:50:38.979796 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.979756    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-567664c68f-69cks"]
Jul 31 14:50:38.982325 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.982303    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:38.985215 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.985195    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 31 14:50:38.985543 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.985525    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 31 14:50:38.985693 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.985677    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 31 14:50:38.985779 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.985765    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-74kb7\""
Jul 31 14:50:38.987226 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.987207    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 31 14:50:38.989801 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.989781    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 31 14:50:38.998400 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:38.998381    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 31 14:50:39.010785 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.010758    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567664c68f-69cks"]
Jul 31 14:50:39.082633 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082603    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-federate-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-metrics-client-ca\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082756 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gzc\" (UniqueName: \"kubernetes.io/projected/b647eca0-5731-4660-9b64-9810c27be9ac-kube-api-access-v9gzc\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082977 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-serving-certs-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.082977 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.082851    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.183887 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.183887 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183891    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gzc\" (UniqueName: \"kubernetes.io/projected/b647eca0-5731-4660-9b64-9810c27be9ac-kube-api-access-v9gzc\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-serving-certs-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.183979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.184020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-federate-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.184039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-metrics-client-ca\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.184918 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.184888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.185035 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.185020    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-serving-certs-ca-bundle\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.185099 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.185062    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b647eca0-5731-4660-9b64-9810c27be9ac-metrics-client-ca\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.186533 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.186501    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-federate-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.186638 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.186507    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.186638 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.186623    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-telemeter-client-tls\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.186717 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.186637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b647eca0-5731-4660-9b64-9810c27be9ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.193591 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.193571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gzc\" (UniqueName: \"kubernetes.io/projected/b647eca0-5731-4660-9b64-9810c27be9ac-kube-api-access-v9gzc\") pod \"telemeter-client-567664c68f-69cks\" (UID: \"b647eca0-5731-4660-9b64-9810c27be9ac\") " pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.291445 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.291376    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567664c68f-69cks"
Jul 31 14:50:39.428875 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.428853    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567664c68f-69cks"]
Jul 31 14:50:39.431451 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:50:39.431416    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb647eca0_5731_4660_9b64_9810c27be9ac.slice/crio-04cd534ae1bdba08762961115f8ef226e9c4d3c9752df42a5b84399f3a7729a0 WatchSource:0}: Error finding container 04cd534ae1bdba08762961115f8ef226e9c4d3c9752df42a5b84399f3a7729a0: Status 404 returned error can't find the container with id 04cd534ae1bdba08762961115f8ef226e9c4d3c9752df42a5b84399f3a7729a0
Jul 31 14:50:39.637200 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:39.637116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567664c68f-69cks" event={"ID":"b647eca0-5731-4660-9b64-9810c27be9ac","Type":"ContainerStarted","Data":"04cd534ae1bdba08762961115f8ef226e9c4d3c9752df42a5b84399f3a7729a0"}
Jul 31 14:50:41.647778 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:41.647741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567664c68f-69cks" event={"ID":"b647eca0-5731-4660-9b64-9810c27be9ac","Type":"ContainerStarted","Data":"20724b80a5f0e38ccdc07bc97f476f4c947479b965c1e1832dd26cd66d54c1e1"}
Jul 31 14:50:41.648251 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:41.647851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567664c68f-69cks" event={"ID":"b647eca0-5731-4660-9b64-9810c27be9ac","Type":"ContainerStarted","Data":"a0d6bdf62275eb8906947e0d323c30072c66582d52d2cbf7943f6ad522e02d80"}
Jul 31 14:50:41.648251 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:41.647889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567664c68f-69cks" event={"ID":"b647eca0-5731-4660-9b64-9810c27be9ac","Type":"ContainerStarted","Data":"9986ab52ae1bf1dc16f1553e6f51825f2f9569e3b100e046406cacb4b11cb689"}
Jul 31 14:50:41.682020 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:50:41.681974    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-567664c68f-69cks" podStartSLOduration=2.178862116 podStartE2EDuration="3.681958967s" totalImagesPullingTime="1.503096851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:50:41.68077152 +0000 UTC m=+184.392746060" watchObservedRunningTime="2026-07-31 14:50:41.681958967 +0000 UTC m=+184.393933495"
Jul 31 14:52:37.654774 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:52:37.654744    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:52:37.663401 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:52:37.663372    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:52:37.669379 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:52:37.669357    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:52:37.670880 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:52:37.670861    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 31 14:52:37.672920 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:52:37.672899    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:54:43.219111 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.219078    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-q726f"]
Jul 31 14:54:43.222360 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.222339    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.225669 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.225646    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-5gv75\""
Jul 31 14:54:43.226712 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.226682    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 31 14:54:43.226881 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.226772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 31 14:54:43.226881 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.226781    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 31 14:54:43.226881 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.226797    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 31 14:54:43.226881 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.226772    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 31 14:54:43.235509 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.235491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-q726f"]
Jul 31 14:54:43.336093 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.336063    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.336269 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.336102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/400145e6-767c-4c84-9510-ea42cb5af8e8-cabundle0\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.336269 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.336187    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf5h6\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-kube-api-access-sf5h6\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.437248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.437212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.437248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.437252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/400145e6-767c-4c84-9510-ea42cb5af8e8-cabundle0\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.437459 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.437296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf5h6\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-kube-api-access-sf5h6\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.437459 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.437358    2566 secret.go:281] references non-existent secret key: ca.crt
Jul 31 14:54:43.437459 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.437377    2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 31 14:54:43.437459 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.437386    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-q726f: references non-existent secret key: ca.crt
Jul 31 14:54:43.437459 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.437454    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates podName:400145e6-767c-4c84-9510-ea42cb5af8e8 nodeName:}" failed. No retries permitted until 2026-07-31 14:54:43.93744012 +0000 UTC m=+426.649414627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates") pod "keda-operator-77f648d694-q726f" (UID: "400145e6-767c-4c84-9510-ea42cb5af8e8") : references non-existent secret key: ca.crt
Jul 31 14:54:43.437900 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.437882    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/400145e6-767c-4c84-9510-ea42cb5af8e8-cabundle0\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.447372 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.447344    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf5h6\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-kube-api-access-sf5h6\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.502692 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.502613    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"]
Jul 31 14:54:43.506094 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.506073    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.514374 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.514350    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 31 14:54:43.521334 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.521309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"]
Jul 31 14:54:43.639452 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.639420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d83d2070-5a42-4050-bc71-77eb12db4cf0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.639614 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.639474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmm8d\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-kube-api-access-cmm8d\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.639614 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.639540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-certificates\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.739986 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.739936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmm8d\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-kube-api-access-cmm8d\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.740181 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.740090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-certificates\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.740248 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.740179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d83d2070-5a42-4050-bc71-77eb12db4cf0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.740610 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.740591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d83d2070-5a42-4050-bc71-77eb12db4cf0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.742660 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.742643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-certificates\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.749583 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.749564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmm8d\" (UniqueName: \"kubernetes.io/projected/d83d2070-5a42-4050-bc71-77eb12db4cf0-kube-api-access-cmm8d\") pod \"keda-metrics-apiserver-55557df8c-9vczr\" (UID: \"d83d2070-5a42-4050-bc71-77eb12db4cf0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.779767 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.779705    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-z5dzw"]
Jul 31 14:54:43.782887 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.782870    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:43.785277 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.785260    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 31 14:54:43.794025 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.794002    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-z5dzw"]
Jul 31 14:54:43.815935 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.815909    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:43.841108 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.841076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:43.841230 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.841165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzt22\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-kube-api-access-vzt22\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:43.940050 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.940017    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"]
Jul 31 14:54:43.942280 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.942258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.942389 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.942298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:43.942389 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.942345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzt22\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-kube-api-access-vzt22\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:43.942488 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.942467    2566 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 31 14:54:43.942540 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.942493    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-z5dzw: secret "keda-admission-webhooks-certs" not found
Jul 31 14:54:43.942587 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:54:43.942546    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates podName:6e44bd0e-185e-403c-ad81-2a302d29ca07 nodeName:}" failed. No retries permitted until 2026-07-31 14:54:44.442529031 +0000 UTC m=+427.154503538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates") pod "keda-admission-56f9798bb4-z5dzw" (UID: "6e44bd0e-185e-403c-ad81-2a302d29ca07") : secret "keda-admission-webhooks-certs" not found
Jul 31 14:54:43.943078 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:54:43.943056    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83d2070_5a42_4050_bc71_77eb12db4cf0.slice/crio-d0c3927cd2e2fad66cbd65fc426199c60b9f2a77a4e38ded70529f8d88c93f3a WatchSource:0}: Error finding container d0c3927cd2e2fad66cbd65fc426199c60b9f2a77a4e38ded70529f8d88c93f3a: Status 404 returned error can't find the container with id d0c3927cd2e2fad66cbd65fc426199c60b9f2a77a4e38ded70529f8d88c93f3a
Jul 31 14:54:43.944279 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.944262    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 14:54:43.944707 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.944689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/400145e6-767c-4c84-9510-ea42cb5af8e8-certificates\") pod \"keda-operator-77f648d694-q726f\" (UID: \"400145e6-767c-4c84-9510-ea42cb5af8e8\") " pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:43.952338 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:43.952320    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzt22\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-kube-api-access-vzt22\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:44.131705 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.131614    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:44.261772 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.261744    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-q726f"]
Jul 31 14:54:44.264265 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:54:44.264238    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400145e6_767c_4c84_9510_ea42cb5af8e8.slice/crio-a5dcd30dacfaf07f1330f9dd51b73e63c547e64895bed9b2b31f9daad46ff7f6 WatchSource:0}: Error finding container a5dcd30dacfaf07f1330f9dd51b73e63c547e64895bed9b2b31f9daad46ff7f6: Status 404 returned error can't find the container with id a5dcd30dacfaf07f1330f9dd51b73e63c547e64895bed9b2b31f9daad46ff7f6
Jul 31 14:54:44.448007 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.447968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:44.450927 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.450901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6e44bd0e-185e-403c-ad81-2a302d29ca07-certificates\") pod \"keda-admission-56f9798bb4-z5dzw\" (UID: \"6e44bd0e-185e-403c-ad81-2a302d29ca07\") " pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:44.575316 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.575277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-q726f" event={"ID":"400145e6-767c-4c84-9510-ea42cb5af8e8","Type":"ContainerStarted","Data":"a5dcd30dacfaf07f1330f9dd51b73e63c547e64895bed9b2b31f9daad46ff7f6"}
Jul 31 14:54:44.576696 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.576664    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr" event={"ID":"d83d2070-5a42-4050-bc71-77eb12db4cf0","Type":"ContainerStarted","Data":"d0c3927cd2e2fad66cbd65fc426199c60b9f2a77a4e38ded70529f8d88c93f3a"}
Jul 31 14:54:44.693411 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.693321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:44.864327 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:44.864301    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-z5dzw"]
Jul 31 14:54:44.866924 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:54:44.866887    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e44bd0e_185e_403c_ad81_2a302d29ca07.slice/crio-3e488985d0cad793fdf9f7d36bfcb546c88584489d1ef20ccecb569c3fd89f4d WatchSource:0}: Error finding container 3e488985d0cad793fdf9f7d36bfcb546c88584489d1ef20ccecb569c3fd89f4d: Status 404 returned error can't find the container with id 3e488985d0cad793fdf9f7d36bfcb546c88584489d1ef20ccecb569c3fd89f4d
Jul 31 14:54:45.584767 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:45.584676    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw" event={"ID":"6e44bd0e-185e-403c-ad81-2a302d29ca07","Type":"ContainerStarted","Data":"3e488985d0cad793fdf9f7d36bfcb546c88584489d1ef20ccecb569c3fd89f4d"}
Jul 31 14:54:48.599679 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.599627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr" event={"ID":"d83d2070-5a42-4050-bc71-77eb12db4cf0","Type":"ContainerStarted","Data":"f936f175a4c1f5efbffbd6198278abeb6ae2bfaf834f05687a006c9e792ee755"}
Jul 31 14:54:48.600136 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.599730    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:54:48.601265 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.601237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-q726f" event={"ID":"400145e6-767c-4c84-9510-ea42cb5af8e8","Type":"ContainerStarted","Data":"a75888791f8b719f2c1fac3d1289aa40361791993ed5713d5872bbe268902923"}
Jul 31 14:54:48.601407 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.601329    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:54:48.602766 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.602741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw" event={"ID":"6e44bd0e-185e-403c-ad81-2a302d29ca07","Type":"ContainerStarted","Data":"6550bc1bb10a179a9eba69c431e72d04639a2d3b1203e5764cb29a041f1634ed"}
Jul 31 14:54:48.602895 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.602862    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:54:48.623104 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.623029    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr" podStartSLOduration=1.16507677 podStartE2EDuration="5.623015569s" totalImagesPullingTime="4.457938799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:54:48.62178497 +0000 UTC m=+431.333759499" watchObservedRunningTime="2026-07-31 14:54:48.623015569 +0000 UTC m=+431.334990095"
Jul 31 14:54:48.644739 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.644698    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw" podStartSLOduration=2.107558072 podStartE2EDuration="5.644683941s" totalImagesPullingTime="3.537125869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:54:48.644525541 +0000 UTC m=+431.356500068" watchObservedRunningTime="2026-07-31 14:54:48.644683941 +0000 UTC m=+431.356658469"
Jul 31 14:54:48.668380 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:48.668324    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-q726f" podStartSLOduration=1.520358296 podStartE2EDuration="5.668305147s" totalImagesPullingTime="4.147946851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:54:48.666406298 +0000 UTC m=+431.378380828" watchObservedRunningTime="2026-07-31 14:54:48.668305147 +0000 UTC m=+431.380279677"
Jul 31 14:54:59.610651 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:54:59.610613    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9vczr"
Jul 31 14:55:09.608668 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:55:09.608588    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-q726f"
Jul 31 14:55:09.609129 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:55:09.608673    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-z5dzw"
Jul 31 14:56:08.304029 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.303995    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:08.311089 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.311051    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.314670 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.314644    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 31 14:56:08.314826 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.314677    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 31 14:56:08.315095 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.315070    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-lsrlc\""
Jul 31 14:56:08.315221 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.315114    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 31 14:56:08.321118 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.321083    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:08.356117 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.356089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.356266 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.356132    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql68z\" (UniqueName: \"kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.457318 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.457279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.457493 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.457327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql68z\" (UniqueName: \"kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.457493 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:56:08.457442    2566 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 31 14:56:08.457562 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:56:08.457522    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert podName:9a305a6a-f851-4c5d-9c3a-7d097834323a nodeName:}" failed. No retries permitted until 2026-07-31 14:56:08.957504 +0000 UTC m=+511.669478507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert") pod "kserve-controller-manager-fcc9dd4db-rjcgq" (UID: "9a305a6a-f851-4c5d-9c3a-7d097834323a") : secret "kserve-webhook-server-cert" not found
Jul 31 14:56:08.467861 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.467840    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql68z\" (UniqueName: \"kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.961643 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.961608    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:08.963956 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:08.963930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") pod \"kserve-controller-manager-fcc9dd4db-rjcgq\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") " pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:09.222471 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:09.222377    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:09.352592 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:09.352566    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:09.354966 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:56:09.354942    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a305a6a_f851_4c5d_9c3a_7d097834323a.slice/crio-37511e1fa7ec74e626ff511701f1ad0fdcbb4d04a5f4f56d029cd0599e531eb3 WatchSource:0}: Error finding container 37511e1fa7ec74e626ff511701f1ad0fdcbb4d04a5f4f56d029cd0599e531eb3: Status 404 returned error can't find the container with id 37511e1fa7ec74e626ff511701f1ad0fdcbb4d04a5f4f56d029cd0599e531eb3
Jul 31 14:56:09.932301 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:09.932265    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" event={"ID":"9a305a6a-f851-4c5d-9c3a-7d097834323a","Type":"ContainerStarted","Data":"37511e1fa7ec74e626ff511701f1ad0fdcbb4d04a5f4f56d029cd0599e531eb3"}
Jul 31 14:56:11.945185 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:11.945149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" event={"ID":"9a305a6a-f851-4c5d-9c3a-7d097834323a","Type":"ContainerStarted","Data":"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"}
Jul 31 14:56:11.945618 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:11.945285    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:11.984414 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:11.984363    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" podStartSLOduration=1.69912335 podStartE2EDuration="3.984347985s" totalImagesPullingTime="2.285224635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:56:11.983959011 +0000 UTC m=+514.695933575" watchObservedRunningTime="2026-07-31 14:56:11.984347985 +0000 UTC m=+514.696322514"
Jul 31 14:56:42.960114 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:42.960080    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:54.694999 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.694963    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:54.695392 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.695154    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" podUID="9a305a6a-f851-4c5d-9c3a-7d097834323a" containerName="manager" containerID="cri-o://2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48" gracePeriod=10
Jul 31 14:56:54.724250 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.724221    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-8n8t2"]
Jul 31 14:56:54.726649 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.726631    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.743998 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.743972    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-8n8t2"]
Jul 31 14:56:54.857740 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.857698    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9nw8\" (UniqueName: \"kubernetes.io/projected/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-kube-api-access-l9nw8\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.857900 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.857776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-cert\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.930164 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.930141    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:54.959259 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.959179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9nw8\" (UniqueName: \"kubernetes.io/projected/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-kube-api-access-l9nw8\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.959409 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.959257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-cert\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.961477 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.961449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-cert\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:54.970298 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:54.970270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9nw8\" (UniqueName: \"kubernetes.io/projected/bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5-kube-api-access-l9nw8\") pod \"kserve-controller-manager-fcc9dd4db-8n8t2\" (UID: \"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5\") " pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:55.060560 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.060522    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") pod \"9a305a6a-f851-4c5d-9c3a-7d097834323a\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") "
Jul 31 14:56:55.060560 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.060568    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql68z\" (UniqueName: \"kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z\") pod \"9a305a6a-f851-4c5d-9c3a-7d097834323a\" (UID: \"9a305a6a-f851-4c5d-9c3a-7d097834323a\") "
Jul 31 14:56:55.062769 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.062742    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z" (OuterVolumeSpecName: "kube-api-access-ql68z") pod "9a305a6a-f851-4c5d-9c3a-7d097834323a" (UID: "9a305a6a-f851-4c5d-9c3a-7d097834323a"). InnerVolumeSpecName "kube-api-access-ql68z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 14:56:55.062769 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.062759    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert" (OuterVolumeSpecName: "cert") pod "9a305a6a-f851-4c5d-9c3a-7d097834323a" (UID: "9a305a6a-f851-4c5d-9c3a-7d097834323a"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 14:56:55.072950 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.072933    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:55.123987 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.123958    2566 generic.go:356] "Generic (PLEG): container finished" podID="9a305a6a-f851-4c5d-9c3a-7d097834323a" containerID="2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48" exitCode=0
Jul 31 14:56:55.124114 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.124003    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq"
Jul 31 14:56:55.124114 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.124048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" event={"ID":"9a305a6a-f851-4c5d-9c3a-7d097834323a","Type":"ContainerDied","Data":"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"}
Jul 31 14:56:55.124219 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.124115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-rjcgq" event={"ID":"9a305a6a-f851-4c5d-9c3a-7d097834323a","Type":"ContainerDied","Data":"37511e1fa7ec74e626ff511701f1ad0fdcbb4d04a5f4f56d029cd0599e531eb3"}
Jul 31 14:56:55.124219 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.124148    2566 scope.go:117] "RemoveContainer" containerID="2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"
Jul 31 14:56:55.142077 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.142054    2566 scope.go:117] "RemoveContainer" containerID="2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"
Jul 31 14:56:55.142364 ip-10-0-132-204 kubenswrapper[2566]: E0731 14:56:55.142345    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48\": container with ID starting with 2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48 not found: ID does not exist" containerID="2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"
Jul 31 14:56:55.142434 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.142378    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48"} err="failed to get container status \"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48\": rpc error: code = NotFound desc = could not find container \"2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48\": container with ID starting with 2f0ffa093eeb666fe3ca00634f595b81fb737e3b738f3439e188a11b99132a48 not found: ID does not exist"
Jul 31 14:56:55.153221 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.153174    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:55.155692 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.155647    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-rjcgq"]
Jul 31 14:56:55.161678 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.161657    2566 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a305a6a-f851-4c5d-9c3a-7d097834323a-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:56:55.161765 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.161681    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql68z\" (UniqueName: \"kubernetes.io/projected/9a305a6a-f851-4c5d-9c3a-7d097834323a-kube-api-access-ql68z\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 14:56:55.205949 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.205920    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-fcc9dd4db-8n8t2"]
Jul 31 14:56:55.208468 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:56:55.208443    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa5a128_b8f3_45fc_b0d0_c907c9ced0d5.slice/crio-a4225400eb9d8740d7ae43c9349345c115decc71b5bb1f618860aad527b4bcd9 WatchSource:0}: Error finding container a4225400eb9d8740d7ae43c9349345c115decc71b5bb1f618860aad527b4bcd9: Status 404 returned error can't find the container with id a4225400eb9d8740d7ae43c9349345c115decc71b5bb1f618860aad527b4bcd9
Jul 31 14:56:55.814012 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:55.813983    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a305a6a-f851-4c5d-9c3a-7d097834323a" path="/var/lib/kubelet/pods/9a305a6a-f851-4c5d-9c3a-7d097834323a/volumes"
Jul 31 14:56:56.130774 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:56.130735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2" event={"ID":"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5","Type":"ContainerStarted","Data":"8581e39e8b0a0f8a54d18942c2737978873e7931a23c94ed22222bbc980a7db9"}
Jul 31 14:56:56.130959 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:56.130795    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2" event={"ID":"bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5","Type":"ContainerStarted","Data":"a4225400eb9d8740d7ae43c9349345c115decc71b5bb1f618860aad527b4bcd9"}
Jul 31 14:56:56.130959 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:56.130876    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:56:56.149982 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:56:56.149935    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2" podStartSLOduration=1.761398996 podStartE2EDuration="2.149920329s" totalImagesPullingTime="388.521333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:56:56.148275019 +0000 UTC m=+558.860249546" watchObservedRunningTime="2026-07-31 14:56:56.149920329 +0000 UTC m=+558.861894862"
Jul 31 14:57:27.139440 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:27.139406    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-fcc9dd4db-8n8t2"
Jul 31 14:57:30.055101 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.055069    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-4k9t6"]
Jul 31 14:57:30.055484 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.055406    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a305a6a-f851-4c5d-9c3a-7d097834323a" containerName="manager"
Jul 31 14:57:30.055484 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.055419    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a305a6a-f851-4c5d-9c3a-7d097834323a" containerName="manager"
Jul 31 14:57:30.055484 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.055484    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a305a6a-f851-4c5d-9c3a-7d097834323a" containerName="manager"
Jul 31 14:57:30.058668 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.058649    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.062919 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.062899    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-z8wct\""
Jul 31 14:57:30.063074 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.062936    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 31 14:57:30.077473 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.077452    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-4k9t6"]
Jul 31 14:57:30.155287 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.155234    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn79q\" (UniqueName: \"kubernetes.io/projected/6954f627-91a3-42b7-b4c0-7acef0c1af7a-kube-api-access-qn79q\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.155452 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.155295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6954f627-91a3-42b7-b4c0-7acef0c1af7a-tls-certs\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.256604 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.256568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn79q\" (UniqueName: \"kubernetes.io/projected/6954f627-91a3-42b7-b4c0-7acef0c1af7a-kube-api-access-qn79q\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.256604 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.256610    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6954f627-91a3-42b7-b4c0-7acef0c1af7a-tls-certs\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.259021 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.258998    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6954f627-91a3-42b7-b4c0-7acef0c1af7a-tls-certs\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.266221 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.266191    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn79q\" (UniqueName: \"kubernetes.io/projected/6954f627-91a3-42b7-b4c0-7acef0c1af7a-kube-api-access-qn79q\") pod \"model-serving-api-7b954ffc95-4k9t6\" (UID: \"6954f627-91a3-42b7-b4c0-7acef0c1af7a\") " pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.369294 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.369211    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:30.498800 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:30.498768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-4k9t6"]
Jul 31 14:57:30.503320 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:57:30.503296    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954f627_91a3_42b7_b4c0_7acef0c1af7a.slice/crio-b5245954db6bebcdfc82c8f4ed025b59b25806b8bd1acc75eeb29b4ef8e043cb WatchSource:0}: Error finding container b5245954db6bebcdfc82c8f4ed025b59b25806b8bd1acc75eeb29b4ef8e043cb: Status 404 returned error can't find the container with id b5245954db6bebcdfc82c8f4ed025b59b25806b8bd1acc75eeb29b4ef8e043cb
Jul 31 14:57:31.284499 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:31.284467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-4k9t6" event={"ID":"6954f627-91a3-42b7-b4c0-7acef0c1af7a","Type":"ContainerStarted","Data":"b5245954db6bebcdfc82c8f4ed025b59b25806b8bd1acc75eeb29b4ef8e043cb"}
Jul 31 14:57:32.289868 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:32.289829    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-4k9t6" event={"ID":"6954f627-91a3-42b7-b4c0-7acef0c1af7a","Type":"ContainerStarted","Data":"631623ac2f9a15a2ace60f275485590bd1ddd652f7d4e9218c01d5473f354aa6"}
Jul 31 14:57:32.290258 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:32.289913    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:57:32.309423 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:32.309328    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-4k9t6" podStartSLOduration=0.77094805 podStartE2EDuration="2.309312083s" totalImagesPullingTime="1.538364033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:57:32.308197788 +0000 UTC m=+595.020172314" watchObservedRunningTime="2026-07-31 14:57:32.309312083 +0000 UTC m=+595.021286612"
Jul 31 14:57:37.689540 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:37.689508    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:57:37.692526 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:37.692501    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:57:37.696131 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:37.696107    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 14:57:37.699295 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:37.699273    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 14:57:43.297948 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:57:43.297918    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-4k9t6"
Jul 31 14:59:18.007405 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.007357    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 14:59:18.010844 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.010825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.014539 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.014509    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 14:59:18.014659 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.014636    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 14:59:18.015558 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.015542    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mhdbw\""
Jul 31 14:59:18.015649 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.015584    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 14:59:18.015649 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.015598    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 14:59:18.015766 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.015648    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-v2-predictor-serving-cert\""
Jul 31 14:59:18.015766 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.015659    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Jul 31 14:59:18.035501 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.035478    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 14:59:18.111958 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.111931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78lkk\" (UniqueName: \"kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.112090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.111976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.112090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.112029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.112090 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.112062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.112189 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.112108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213313 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78lkk\" (UniqueName: \"kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213450 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213450 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213450 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213450 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.213898 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.213880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.214146 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.214125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.214146 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.214137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.215843 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.215798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.221307 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.221288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78lkk\" (UniqueName: \"kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk\") pod \"isvc-canary-rollback-v2-predictor-758954f9f-zcg8l\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.320848 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.320757    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:18.452791 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.452766    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 14:59:18.455262 ip-10-0-132-204 kubenswrapper[2566]: W0731 14:59:18.455232    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812cb9df_f2b7_49f2_a06d_0c0c3bdc515e.slice/crio-0c5f92b1850fbf23cfc83adfc0bcd4d00d1102cfe3cf3238ddfccec97c69206c WatchSource:0}: Error finding container 0c5f92b1850fbf23cfc83adfc0bcd4d00d1102cfe3cf3238ddfccec97c69206c: Status 404 returned error can't find the container with id 0c5f92b1850fbf23cfc83adfc0bcd4d00d1102cfe3cf3238ddfccec97c69206c
Jul 31 14:59:18.749448 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:18.749414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerStarted","Data":"0c5f92b1850fbf23cfc83adfc0bcd4d00d1102cfe3cf3238ddfccec97c69206c"}
Jul 31 14:59:21.768953 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:21.768915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerStarted","Data":"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"}
Jul 31 14:59:25.787440 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:25.787408    2566 generic.go:356] "Generic (PLEG): container finished" podID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerID="849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c" exitCode=0
Jul 31 14:59:25.787776 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:25.787478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerDied","Data":"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"}
Jul 31 14:59:38.863460 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:38.863418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerStarted","Data":"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"}
Jul 31 14:59:40.875429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:40.875356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerStarted","Data":"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"}
Jul 31 14:59:40.875429 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:40.875414    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:40.875788 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:40.875462    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:40.876611 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:40.876583    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 14:59:40.899906 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:40.899858    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podStartSLOduration=2.484652338 podStartE2EDuration="23.899841294s" totalImagesPullingTime="17.608893013s" totalInitContainerRuntime="3.806295943s" isStatefulPod=false podCreationTimestamp="2026-07-31 14:59:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 14:59:40.897584391 +0000 UTC m=+723.609558955" watchObservedRunningTime="2026-07-31 14:59:40.899841294 +0000 UTC m=+723.611815824"
Jul 31 14:59:41.879972 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:41.879935    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 14:59:46.885057 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:46.885028    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 14:59:46.885541 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:46.885515    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 14:59:56.886086 ip-10-0-132-204 kubenswrapper[2566]: I0731 14:59:56.886050    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 15:00:06.886154 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:00:06.886106    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 15:00:16.886331 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:00:16.886290    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 15:00:26.885761 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:00:26.885715    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 15:00:36.886116 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:00:36.886071    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 31 15:00:46.886457 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:00:46.886427    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 15:01:07.575291 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.575211    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 15:01:07.575728 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.575481    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container" containerID="cri-o://01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9" gracePeriod=30
Jul 31 15:01:07.575728 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.575522    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kube-rbac-proxy" containerID="cri-o://459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1" gracePeriod=30
Jul 31 15:01:07.653184 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.653154    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:01:07.655913 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.655892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.658320 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.658295    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Jul 31 15:01:07.666159 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.666135    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:01:07.729685 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.729656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.729874 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.729696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.729874 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.729731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.729874 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.729827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.729874 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.729870    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lpz\" (UniqueName: \"kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831137 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831137 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lpz\" (UniqueName: \"kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831302 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831345 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831395 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831493 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831473    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831747 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831731    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.831892 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.831874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.833595 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.833580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.841027 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.841010    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lpz\" (UniqueName: \"kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz\") pod \"isvc-canary-rollback-predictor-7f985c86-6cgth\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:07.966478 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:07.966443    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:08.093236 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:08.093212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:01:08.095189 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:01:08.095166    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfd11d8_fa0b_46e8_81a7_640a6ec71a68.slice/crio-2b435c96e315e07310f61d8bc5a63d741a45fe48cf3c495d5dd7307bed96a834 WatchSource:0}: Error finding container 2b435c96e315e07310f61d8bc5a63d741a45fe48cf3c495d5dd7307bed96a834: Status 404 returned error can't find the container with id 2b435c96e315e07310f61d8bc5a63d741a45fe48cf3c495d5dd7307bed96a834
Jul 31 15:01:08.261010 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:08.260978    2566 generic.go:356] "Generic (PLEG): container finished" podID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerID="459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1" exitCode=2
Jul 31 15:01:08.261189 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:08.261052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerDied","Data":"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"}
Jul 31 15:01:08.262672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:08.262641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerStarted","Data":"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"}
Jul 31 15:01:08.262838 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:08.262700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerStarted","Data":"2b435c96e315e07310f61d8bc5a63d741a45fe48cf3c495d5dd7307bed96a834"}
Jul 31 15:01:11.881096 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:11.881051    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Jul 31 15:01:12.125295 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.125271    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 15:01:12.272863 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.272785    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78lkk\" (UniqueName: \"kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk\") pod \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") "
Jul 31 15:01:12.273020 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.272878    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert\") pod \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") "
Jul 31 15:01:12.273020 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.272980    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls\") pod \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") "
Jul 31 15:01:12.273020 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.273009    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location\") pod \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") "
Jul 31 15:01:12.273156 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.273053    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config\") pod \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\" (UID: \"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e\") "
Jul 31 15:01:12.273328 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.273298    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" (UID: "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:01:12.273471 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.273328    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" (UID: "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:01:12.273471 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.273392    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" (UID: "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:01:12.275031 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.275005    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk" (OuterVolumeSpecName: "kube-api-access-78lkk") pod "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" (UID: "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e"). InnerVolumeSpecName "kube-api-access-78lkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:01:12.275136 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.275075    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" (UID: "812cb9df-f2b7-49f2-a06d-0c0c3bdc515e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:01:12.281511 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.281488    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerID="fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5" exitCode=0
Jul 31 15:01:12.281598 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.281558    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerDied","Data":"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"}
Jul 31 15:01:12.283524 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.283503    2566 generic.go:356] "Generic (PLEG): container finished" podID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerID="01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9" exitCode=0
Jul 31 15:01:12.283614 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.283556    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"
Jul 31 15:01:12.283614 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.283579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerDied","Data":"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"}
Jul 31 15:01:12.283699 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.283629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l" event={"ID":"812cb9df-f2b7-49f2-a06d-0c0c3bdc515e","Type":"ContainerDied","Data":"0c5f92b1850fbf23cfc83adfc0bcd4d00d1102cfe3cf3238ddfccec97c69206c"}
Jul 31 15:01:12.283699 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.283667    2566 scope.go:117] "RemoveContainer" containerID="459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"
Jul 31 15:01:12.296109 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.296089    2566 scope.go:117] "RemoveContainer" containerID="01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"
Jul 31 15:01:12.304511 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.304485    2566 scope.go:117] "RemoveContainer" containerID="849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"
Jul 31 15:01:12.317590 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.317567    2566 scope.go:117] "RemoveContainer" containerID="459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"
Jul 31 15:01:12.317696 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.317648    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 15:01:12.318043 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:01:12.318017    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1\": container with ID starting with 459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1 not found: ID does not exist" containerID="459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"
Jul 31 15:01:12.318128 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.318056    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1"} err="failed to get container status \"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1\": rpc error: code = NotFound desc = could not find container \"459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1\": container with ID starting with 459cad246c03e5f9294bd5c6281547e254e24dea3d54bf46d2c480d695da69d1 not found: ID does not exist"
Jul 31 15:01:12.318128 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.318079    2566 scope.go:117] "RemoveContainer" containerID="01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"
Jul 31 15:01:12.318415 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:01:12.318394    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9\": container with ID starting with 01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9 not found: ID does not exist" containerID="01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"
Jul 31 15:01:12.318495 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.318420    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9"} err="failed to get container status \"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9\": rpc error: code = NotFound desc = could not find container \"01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9\": container with ID starting with 01c0334711b62f8e9c75ce740b73a6651de915b369bdb848ffda7a1b850a4ed9 not found: ID does not exist"
Jul 31 15:01:12.318495 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.318436    2566 scope.go:117] "RemoveContainer" containerID="849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"
Jul 31 15:01:12.318741 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:01:12.318719    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c\": container with ID starting with 849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c not found: ID does not exist" containerID="849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"
Jul 31 15:01:12.318798 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.318750    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c"} err="failed to get container status \"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c\": rpc error: code = NotFound desc = could not find container \"849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c\": container with ID starting with 849e40afb2032d0f51dcfce29652992082af30cb388d4d39f41a47588e47747c not found: ID does not exist"
Jul 31 15:01:12.319483 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.319461    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-v2-predictor-758954f9f-zcg8l"]
Jul 31 15:01:12.373777 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.373749    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:01:12.373777 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.373778    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:01:12.373959 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.373791    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:01:12.373959 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.373820    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:01:12.373959 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.373836    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78lkk\" (UniqueName: \"kubernetes.io/projected/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e-kube-api-access-78lkk\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:01:12.911185 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911152    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911501    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="storage-initializer"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911513    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="storage-initializer"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911525    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911531    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911536    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kube-rbac-proxy"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911542    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kube-rbac-proxy"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911607    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kube-rbac-proxy"
Jul 31 15:01:12.911672 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.911617    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" containerName="kserve-container"
Jul 31 15:01:12.913995 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.913976    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:12.916628 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.916609    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-b123a-predictor-serving-cert\""
Jul 31 15:01:12.916723 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.916712    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-b123a-kube-rbac-proxy-sar-config\""
Jul 31 15:01:12.928384 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.928360    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:01:12.979139 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.979107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:12.979301 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.979158    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:12.979301 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.979245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt77x\" (UniqueName: \"kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:12.979369 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.979300    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:12.979398 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:12.979364    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.080508 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.080471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt77x\" (UniqueName: \"kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.080716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.080516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.080716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.080548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.080716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.080583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.080716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.080616    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.081070 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.081038    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.081327 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.081297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.081422 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.081297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.083059 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.083027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.088912 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.088891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt77x\" (UniqueName: \"kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x\") pod \"isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.231941 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.231906    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:13.294549 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.294511    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerStarted","Data":"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"}
Jul 31 15:01:13.295201 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.294628    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerStarted","Data":"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"}
Jul 31 15:01:13.295201 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.294940    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:13.295201 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.294988    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:13.296583 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.296557    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:13.319620 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.319569    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podStartSLOduration=2.544146277 podStartE2EDuration="6.319554215s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.775407938s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:01:13.317717995 +0000 UTC m=+816.029692569" watchObservedRunningTime="2026-07-31 15:01:13.319554215 +0000 UTC m=+816.031528742"
Jul 31 15:01:13.366727 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.366701    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:01:13.369182 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:01:13.369160    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4720e0cb_7d65_4b52_bfcb_9e402115d3d9.slice/crio-e54208f956de3f6edd9af833febc7c94e8c7f44aa18dfb605c7591f2f7599b6d WatchSource:0}: Error finding container e54208f956de3f6edd9af833febc7c94e8c7f44aa18dfb605c7591f2f7599b6d: Status 404 returned error can't find the container with id e54208f956de3f6edd9af833febc7c94e8c7f44aa18dfb605c7591f2f7599b6d
Jul 31 15:01:13.815318 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:13.815281    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="812cb9df-f2b7-49f2-a06d-0c0c3bdc515e" path="/var/lib/kubelet/pods/812cb9df-f2b7-49f2-a06d-0c0c3bdc515e/volumes"
Jul 31 15:01:14.300492 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:14.300453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerStarted","Data":"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"}
Jul 31 15:01:14.300992 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:14.300523    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerStarted","Data":"e54208f956de3f6edd9af833febc7c94e8c7f44aa18dfb605c7591f2f7599b6d"}
Jul 31 15:01:14.300992 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:14.300770    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:18.319384 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:18.319353    2566 generic.go:356] "Generic (PLEG): container finished" podID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerID="9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388" exitCode=0
Jul 31 15:01:18.319731 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:18.319430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerDied","Data":"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"}
Jul 31 15:01:19.305635 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.305606    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:01:19.306130 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.306102    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:19.326307 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.326275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerStarted","Data":"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"}
Jul 31 15:01:19.326713 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.326336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerStarted","Data":"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"}
Jul 31 15:01:19.326713 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.326550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:19.352020 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:19.351969    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podStartSLOduration=3.410898989 podStartE2EDuration="7.351953509s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.94105452s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:01:19.350738165 +0000 UTC m=+822.062712704" watchObservedRunningTime="2026-07-31 15:01:19.351953509 +0000 UTC m=+822.063928039"
Jul 31 15:01:20.331004 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:20.330974    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:20.332364 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:20.332335    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:21.334673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:21.334631    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:26.339318 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:26.339291    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:01:26.339801 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:26.339775    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:29.306507 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:29.306470    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:36.339973 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:36.339931    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:39.306702 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:39.306659    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:46.339853 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:46.339792    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:49.306565 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:49.306523    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:01:56.340321 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:56.340279    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:01:59.306655 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:01:59.306617    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:02:06.339834 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:06.339777    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:02:09.306728 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:09.306691    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:02:16.340372 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:16.340330    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:02:19.306197 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:19.306156    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:02:26.339758 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:26.339721    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 31 15:02:28.809721 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:28.809688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:02:29.307948 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:29.307917    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:02:37.721601 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.721525    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:02:37.726879 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.726849    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:02:37.729149 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.729127    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:02:37.733352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.733328    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:02:37.741646 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.741626    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:02:37.741910 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.741865    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" containerID="cri-o://8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1" gracePeriod=30
Jul 31 15:02:37.742003 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:37.741876    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kube-rbac-proxy" containerID="cri-o://2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e" gracePeriod=30
Jul 31 15:02:38.683826 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:38.683772    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerID="2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e" exitCode=2
Jul 31 15:02:38.684025 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:38.683845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerDied","Data":"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"}
Jul 31 15:02:39.301819 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:39.301772    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 31 15:02:39.306217 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:39.306192    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 31 15:02:42.175028 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.175005    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:02:42.323013 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.322944    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config\") pod \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") "
Jul 31 15:02:42.323155 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323038    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67lpz\" (UniqueName: \"kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz\") pod \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") "
Jul 31 15:02:42.323155 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323069    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location\") pod \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") "
Jul 31 15:02:42.323155 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls\") pod \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") "
Jul 31 15:02:42.323155 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323116    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert\") pod \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\" (UID: \"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68\") "
Jul 31 15:02:42.323438 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323415    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" (UID: "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:02:42.323522 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323415    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" (UID: "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:02:42.323522 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.323470    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" (UID: "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:02:42.325095 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.325073    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz" (OuterVolumeSpecName: "kube-api-access-67lpz") pod "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" (UID: "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68"). InnerVolumeSpecName "kube-api-access-67lpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:02:42.325179 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.325120    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" (UID: "ebfd11d8-fa0b-46e8-81a7-640a6ec71a68"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:02:42.424506 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.424475    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67lpz\" (UniqueName: \"kubernetes.io/projected/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-api-access-67lpz\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:02:42.424506 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.424501    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:02:42.424506 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.424513    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:02:42.424702 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.424522    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:02:42.424702 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.424532    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:02:42.704025 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.703992    2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerID="8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1" exitCode=0
Jul 31 15:02:42.704213 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.704053    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"
Jul 31 15:02:42.704213 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.704058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerDied","Data":"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"}
Jul 31 15:02:42.704213 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.704177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth" event={"ID":"ebfd11d8-fa0b-46e8-81a7-640a6ec71a68","Type":"ContainerDied","Data":"2b435c96e315e07310f61d8bc5a63d741a45fe48cf3c495d5dd7307bed96a834"}
Jul 31 15:02:42.704213 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.704204    2566 scope.go:117] "RemoveContainer" containerID="2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"
Jul 31 15:02:42.713233 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.713217    2566 scope.go:117] "RemoveContainer" containerID="8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"
Jul 31 15:02:42.720678 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.720660    2566 scope.go:117] "RemoveContainer" containerID="fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"
Jul 31 15:02:42.729728 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.729698    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:02:42.731303 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.731282    2566 scope.go:117] "RemoveContainer" containerID="2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"
Jul 31 15:02:42.731630 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:02:42.731609    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e\": container with ID starting with 2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e not found: ID does not exist" containerID="2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"
Jul 31 15:02:42.731698 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.731638    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e"} err="failed to get container status \"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e\": rpc error: code = NotFound desc = could not find container \"2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e\": container with ID starting with 2c80fbaa3cb73d53be97d1678c2d8368a4041732be5afdceeeeae46a5b47cc1e not found: ID does not exist"
Jul 31 15:02:42.731698 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.731655    2566 scope.go:117] "RemoveContainer" containerID="8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"
Jul 31 15:02:42.732479 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:02:42.731943    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1\": container with ID starting with 8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1 not found: ID does not exist" containerID="8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"
Jul 31 15:02:42.732479 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.732010    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1"} err="failed to get container status \"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1\": rpc error: code = NotFound desc = could not find container \"8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1\": container with ID starting with 8d66a17e435f72c603088a037845c83aab81b13406dfaf96e908b96fc4de77c1 not found: ID does not exist"
Jul 31 15:02:42.732479 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.732032    2566 scope.go:117] "RemoveContainer" containerID="fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"
Jul 31 15:02:42.732749 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:02:42.732725    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5\": container with ID starting with fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5 not found: ID does not exist" containerID="fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"
Jul 31 15:02:42.732852 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.732754    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5"} err="failed to get container status \"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5\": rpc error: code = NotFound desc = could not find container \"fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5\": container with ID starting with fde09f7c0669800c06d91c3472d5b472db43d3131695e85f5910cd3bdad23bb5 not found: ID does not exist"
Jul 31 15:02:42.734990 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.734972    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-7f985c86-6cgth"]
Jul 31 15:02:42.976339 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976263    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:02:42.976640 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976627    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="storage-initializer"
Jul 31 15:02:42.976685 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976642    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="storage-initializer"
Jul 31 15:02:42.976685 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976667    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container"
Jul 31 15:02:42.976685 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976673    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container"
Jul 31 15:02:42.976685 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976684    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kube-rbac-proxy"
Jul 31 15:02:42.976802 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976690    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kube-rbac-proxy"
Jul 31 15:02:42.976802 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976745    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kserve-container"
Jul 31 15:02:42.976802 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.976753    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" containerName="kube-rbac-proxy"
Jul 31 15:02:42.980140 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.980119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:42.982899 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.982868    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-v2-predictor-serving-cert\""
Jul 31 15:02:42.983019 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.982927    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Jul 31 15:02:42.991316 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:42.991293    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:02:43.031282 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.031249    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw8td\" (UniqueName: \"kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.031282 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.031285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.031461 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.031338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.031461 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.031352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.031461 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.031430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.131944 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.131874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132092 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132035    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw8td\" (UniqueName: \"kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132175 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132175 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132161    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132282 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132185    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132476 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:02:43.132439    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-serving-cert: secret "isvc-canary-stop-v2-predictor-serving-cert" not found
Jul 31 15:02:43.132645 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:02:43.132505    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls podName:be948750-737d-4a22-ab13-73489de01154 nodeName:}" failed. No retries permitted until 2026-07-31 15:02:43.632487116 +0000 UTC m=+906.344461623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls") pod "isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" (UID: "be948750-737d-4a22-ab13-73489de01154") : secret "isvc-canary-stop-v2-predictor-serving-cert" not found
Jul 31 15:02:43.132732 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132876 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132857    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.132991 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.132973    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.143320 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.143293    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw8td\" (UniqueName: \"kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.634789 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.634761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.637165 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.637134    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-87f57fc87-ln4xq\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:43.814293 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.814260    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebfd11d8-fa0b-46e8-81a7-640a6ec71a68" path="/var/lib/kubelet/pods/ebfd11d8-fa0b-46e8-81a7-640a6ec71a68/volumes"
Jul 31 15:02:43.890502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:43.890435    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:44.023414 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:44.023389    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:02:44.025685 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:02:44.025652    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe948750_737d_4a22_ab13_73489de01154.slice/crio-bea848a20639bee5f553507c608040a36019063953e74205a819a5055f01f3c6 WatchSource:0}: Error finding container bea848a20639bee5f553507c608040a36019063953e74205a819a5055f01f3c6: Status 404 returned error can't find the container with id bea848a20639bee5f553507c608040a36019063953e74205a819a5055f01f3c6
Jul 31 15:02:44.714590 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:44.714556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerStarted","Data":"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"}
Jul 31 15:02:44.714977 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:44.714617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerStarted","Data":"bea848a20639bee5f553507c608040a36019063953e74205a819a5055f01f3c6"}
Jul 31 15:02:48.734853 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:48.734802    2566 generic.go:356] "Generic (PLEG): container finished" podID="be948750-737d-4a22-ab13-73489de01154" containerID="df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d" exitCode=0
Jul 31 15:02:48.735321 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:48.734877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerDied","Data":"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"}
Jul 31 15:02:49.741504 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:49.741471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerStarted","Data":"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"}
Jul 31 15:02:49.741905 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:49.741531    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerStarted","Data":"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"}
Jul 31 15:02:49.741905 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:49.741707    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:49.765181 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:49.765130    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podStartSLOduration=3.928742589 podStartE2EDuration="7.765111667s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.836369078s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:02:49.763755197 +0000 UTC m=+912.475729737" watchObservedRunningTime="2026-07-31 15:02:49.765111667 +0000 UTC m=+912.477086203"
Jul 31 15:02:50.746618 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:50.746581    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:50.747866 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:50.747839    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:02:51.750799 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:51.750757    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:02:56.755649 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:56.755622    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:02:56.756258 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:02:56.756228    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:03.144388 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:03.144352    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:03:03.144743 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:03.144651    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container" containerID="cri-o://94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285" gracePeriod=30
Jul 31 15:03:03.144743 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:03.144689    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kube-rbac-proxy" containerID="cri-o://bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f" gracePeriod=30
Jul 31 15:03:03.806957 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:03.806917    2566 generic.go:356] "Generic (PLEG): container finished" podID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerID="bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f" exitCode=2
Jul 31 15:03:03.807122 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:03.806989    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerDied","Data":"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"}
Jul 31 15:03:06.335633 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:06.335587    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused"
Jul 31 15:03:06.756902 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:06.756864    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:07.693144 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.693113    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:03:07.825470 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.825381    2566 generic.go:356] "Generic (PLEG): container finished" podID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerID="94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285" exitCode=0
Jul 31 15:03:07.825470 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.825436    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"
Jul 31 15:03:07.825645 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.825467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerDied","Data":"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"}
Jul 31 15:03:07.825645 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.825523    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4" event={"ID":"4720e0cb-7d65-4b52-bfcb-9e402115d3d9","Type":"ContainerDied","Data":"e54208f956de3f6edd9af833febc7c94e8c7f44aa18dfb605c7591f2f7599b6d"}
Jul 31 15:03:07.825645 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.825550    2566 scope.go:117] "RemoveContainer" containerID="bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"
Jul 31 15:03:07.833186 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.833168    2566 scope.go:117] "RemoveContainer" containerID="94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"
Jul 31 15:03:07.835677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.835656    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt77x\" (UniqueName: \"kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x\") pod \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") "
Jul 31 15:03:07.835775 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.835697    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config\") pod \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") "
Jul 31 15:03:07.835860 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.835823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert\") pod \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") "
Jul 31 15:03:07.835860 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.835851    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location\") pod \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") "
Jul 31 15:03:07.835977 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.835873    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls\") pod \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\" (UID: \"4720e0cb-7d65-4b52-bfcb-9e402115d3d9\") "
Jul 31 15:03:07.836387 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.836095    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4720e0cb-7d65-4b52-bfcb-9e402115d3d9" (UID: "4720e0cb-7d65-4b52-bfcb-9e402115d3d9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:03:07.836387 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.836141    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4720e0cb-7d65-4b52-bfcb-9e402115d3d9" (UID: "4720e0cb-7d65-4b52-bfcb-9e402115d3d9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:03:07.836387 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.836138    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4720e0cb-7d65-4b52-bfcb-9e402115d3d9" (UID: "4720e0cb-7d65-4b52-bfcb-9e402115d3d9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:03:07.837856 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.837830    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x" (OuterVolumeSpecName: "kube-api-access-wt77x") pod "4720e0cb-7d65-4b52-bfcb-9e402115d3d9" (UID: "4720e0cb-7d65-4b52-bfcb-9e402115d3d9"). InnerVolumeSpecName "kube-api-access-wt77x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:03:07.837992 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.837970    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4720e0cb-7d65-4b52-bfcb-9e402115d3d9" (UID: "4720e0cb-7d65-4b52-bfcb-9e402115d3d9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:03:07.840860 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.840841    2566 scope.go:117] "RemoveContainer" containerID="9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"
Jul 31 15:03:07.852800 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.852780    2566 scope.go:117] "RemoveContainer" containerID="bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"
Jul 31 15:03:07.853054 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:03:07.853036    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f\": container with ID starting with bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f not found: ID does not exist" containerID="bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"
Jul 31 15:03:07.853115 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.853062    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f"} err="failed to get container status \"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f\": rpc error: code = NotFound desc = could not find container \"bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f\": container with ID starting with bad62b88b845ad4e20ef0928f4865998a786c00e35e2035a474061dbf9a6017f not found: ID does not exist"
Jul 31 15:03:07.853115 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.853082    2566 scope.go:117] "RemoveContainer" containerID="94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"
Jul 31 15:03:07.853289 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:03:07.853272    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285\": container with ID starting with 94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285 not found: ID does not exist" containerID="94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"
Jul 31 15:03:07.853351 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.853298    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285"} err="failed to get container status \"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285\": rpc error: code = NotFound desc = could not find container \"94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285\": container with ID starting with 94c1d94b78a9970ed99e78e5390ef93abef2f005b8291c588f4e3a91eb273285 not found: ID does not exist"
Jul 31 15:03:07.853351 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.853319    2566 scope.go:117] "RemoveContainer" containerID="9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"
Jul 31 15:03:07.853555 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:03:07.853538    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388\": container with ID starting with 9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388 not found: ID does not exist" containerID="9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"
Jul 31 15:03:07.853588 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.853559    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388"} err="failed to get container status \"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388\": rpc error: code = NotFound desc = could not find container \"9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388\": container with ID starting with 9276f06378d0183fde6fe01bab4d61bd5f5cb99e5133c3176743b2f81708e388 not found: ID does not exist"
Jul 31 15:03:07.936544 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.936523    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:03:07.936544 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.936543    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:03:07.936673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.936556    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:03:07.936673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.936571    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt77x\" (UniqueName: \"kubernetes.io/projected/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-api-access-wt77x\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:03:07.936673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:07.936585    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4720e0cb-7d65-4b52-bfcb-9e402115d3d9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:03:08.148393 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:08.148365    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:03:08.153365 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:08.153340    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-b123a-predictor-866f4b64d7-r4nz4"]
Jul 31 15:03:09.814680 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:09.814647    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" path="/var/lib/kubelet/pods/4720e0cb-7d65-4b52-bfcb-9e402115d3d9/volumes"
Jul 31 15:03:16.756413 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:16.756370    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:26.756952 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:26.756912    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:36.757020 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:36.756979    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:46.756359 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:46.756317    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:56.756164 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:56.756122    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 31 15:03:58.809744 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:03:58.809717    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:04:22.973195 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:22.973164    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:04:22.974787 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:22.973379    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container" containerID="cri-o://736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667" gracePeriod=30
Jul 31 15:04:22.974787 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:22.973441    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kube-rbac-proxy" containerID="cri-o://18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa" gracePeriod=30
Jul 31 15:04:23.170291 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:23.170261    2566 generic.go:356] "Generic (PLEG): container finished" podID="be948750-737d-4a22-ab13-73489de01154" containerID="18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa" exitCode=2
Jul 31 15:04:23.170436 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:23.170334    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerDied","Data":"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"}
Jul 31 15:04:26.751436 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:26.751396    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 31 15:04:27.108799 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.108774    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:04:27.186238 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186207    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config\") pod \"be948750-737d-4a22-ab13-73489de01154\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") "
Jul 31 15:04:27.186397 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186244    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") pod \"be948750-737d-4a22-ab13-73489de01154\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") "
Jul 31 15:04:27.186397 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186269    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert\") pod \"be948750-737d-4a22-ab13-73489de01154\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") "
Jul 31 15:04:27.186397 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186289    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw8td\" (UniqueName: \"kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td\") pod \"be948750-737d-4a22-ab13-73489de01154\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") "
Jul 31 15:04:27.186397 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186384    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location\") pod \"be948750-737d-4a22-ab13-73489de01154\" (UID: \"be948750-737d-4a22-ab13-73489de01154\") "
Jul 31 15:04:27.186646 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186604    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "be948750-737d-4a22-ab13-73489de01154" (UID: "be948750-737d-4a22-ab13-73489de01154"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:04:27.186755 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186666    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "be948750-737d-4a22-ab13-73489de01154" (UID: "be948750-737d-4a22-ab13-73489de01154"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:04:27.186755 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.186683    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "be948750-737d-4a22-ab13-73489de01154" (UID: "be948750-737d-4a22-ab13-73489de01154"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:04:27.188385 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.188363    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "be948750-737d-4a22-ab13-73489de01154" (UID: "be948750-737d-4a22-ab13-73489de01154"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:04:27.188494 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.188395    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td" (OuterVolumeSpecName: "kube-api-access-gw8td") pod "be948750-737d-4a22-ab13-73489de01154" (UID: "be948750-737d-4a22-ab13-73489de01154"). InnerVolumeSpecName "kube-api-access-gw8td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:04:27.189430 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.189403    2566 generic.go:356] "Generic (PLEG): container finished" podID="be948750-737d-4a22-ab13-73489de01154" containerID="736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667" exitCode=0
Jul 31 15:04:27.189524 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.189446    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerDied","Data":"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"}
Jul 31 15:04:27.189524 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.189468    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"
Jul 31 15:04:27.189524 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.189512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq" event={"ID":"be948750-737d-4a22-ab13-73489de01154","Type":"ContainerDied","Data":"bea848a20639bee5f553507c608040a36019063953e74205a819a5055f01f3c6"}
Jul 31 15:04:27.189668 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.189553    2566 scope.go:117] "RemoveContainer" containerID="18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"
Jul 31 15:04:27.203638 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.203620    2566 scope.go:117] "RemoveContainer" containerID="736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"
Jul 31 15:04:27.210424 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.210407    2566 scope.go:117] "RemoveContainer" containerID="df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"
Jul 31 15:04:27.216814 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.216780    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:04:27.218847 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.218831    2566 scope.go:117] "RemoveContainer" containerID="18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"
Jul 31 15:04:27.219089 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:04:27.219070    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa\": container with ID starting with 18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa not found: ID does not exist" containerID="18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"
Jul 31 15:04:27.219137 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.219095    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa"} err="failed to get container status \"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa\": rpc error: code = NotFound desc = could not find container \"18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa\": container with ID starting with 18ee6e83ffe8b4ad87df34a27ec83528a35eab607ead85a180b13d73469561aa not found: ID does not exist"
Jul 31 15:04:27.219137 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.219113    2566 scope.go:117] "RemoveContainer" containerID="736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"
Jul 31 15:04:27.219366 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:04:27.219346    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667\": container with ID starting with 736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667 not found: ID does not exist" containerID="736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"
Jul 31 15:04:27.219431 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.219376    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667"} err="failed to get container status \"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667\": rpc error: code = NotFound desc = could not find container \"736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667\": container with ID starting with 736cdd77453f6ff3280f5248dd7a6967be61b8b5e460284e722c1fa39406c667 not found: ID does not exist"
Jul 31 15:04:27.219431 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.219398    2566 scope.go:117] "RemoveContainer" containerID="df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"
Jul 31 15:04:27.219646 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:04:27.219623    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d\": container with ID starting with df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d not found: ID does not exist" containerID="df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"
Jul 31 15:04:27.219747 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.219648    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d"} err="failed to get container status \"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d\": rpc error: code = NotFound desc = could not find container \"df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d\": container with ID starting with df2d187523303f6efde14937e1f9eddfa80a7175cbec2b00a4528abedd0bdf8d not found: ID does not exist"
Jul 31 15:04:27.220331 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.220310    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-87f57fc87-ln4xq"]
Jul 31 15:04:27.287935 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.287886    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be948750-737d-4a22-ab13-73489de01154-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:04:27.287935 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.287906    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:04:27.287935 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.287917    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be948750-737d-4a22-ab13-73489de01154-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:04:27.287935 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.287925    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be948750-737d-4a22-ab13-73489de01154-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:04:27.287935 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.287933    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw8td\" (UniqueName: \"kubernetes.io/projected/be948750-737d-4a22-ab13-73489de01154-kube-api-access-gw8td\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:04:27.814471 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:27.814436    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be948750-737d-4a22-ab13-73489de01154" path="/var/lib/kubelet/pods/be948750-737d-4a22-ab13-73489de01154/volumes"
Jul 31 15:04:33.254005 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.253970    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254298    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="storage-initializer"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254308    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="storage-initializer"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254323    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254328    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254337    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kube-rbac-proxy"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254342    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kube-rbac-proxy"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254350    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254355    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254361    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="storage-initializer"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254366    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="storage-initializer"
Jul 31 15:04:33.254367 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254371    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kube-rbac-proxy"
Jul 31 15:04:33.254673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254376    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kube-rbac-proxy"
Jul 31 15:04:33.254673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254437    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kube-rbac-proxy"
Jul 31 15:04:33.254673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254446    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kserve-container"
Jul 31 15:04:33.254673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254452    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="be948750-737d-4a22-ab13-73489de01154" containerName="kserve-container"
Jul 31 15:04:33.254673 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.254459    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4720e0cb-7d65-4b52-bfcb-9e402115d3d9" containerName="kube-rbac-proxy"
Jul 31 15:04:33.257559 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.257542    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.261753 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.261725    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 15:04:33.261753 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.261747    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 15:04:33.261962 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.261792    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-74ffcf-predictor-serving-cert\""
Jul 31 15:04:33.261962 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.261859    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-74ffcf-kube-rbac-proxy-sar-config\""
Jul 31 15:04:33.262058 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.262020    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 15:04:33.262169 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.262153    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mhdbw\""
Jul 31 15:04:33.262257 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.262241    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 15:04:33.270561 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.270539    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:04:33.330033 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.330007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.330179 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.330039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.330179 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.330076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.330295 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.330189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.330295 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.330240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb9hb\" (UniqueName: \"kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431144 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431268 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb9hb\" (UniqueName: \"kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431268 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431268 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431268 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431253    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431611 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431593    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431901 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.431982 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.431898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.433769 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.433745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.441753 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.441733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb9hb\" (UniqueName: \"kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb\") pod \"isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") " pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.567691 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.567628    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:33.698890 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:33.698866    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:04:33.701553 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:04:33.701524    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91aefa0_8a19_4a75_b685_57626fd328d8.slice/crio-cbe8000e8111bcfaec51cd262411f14ad1d23482826f1a40b2bf5b2903f44566 WatchSource:0}: Error finding container cbe8000e8111bcfaec51cd262411f14ad1d23482826f1a40b2bf5b2903f44566: Status 404 returned error can't find the container with id cbe8000e8111bcfaec51cd262411f14ad1d23482826f1a40b2bf5b2903f44566
Jul 31 15:04:34.226949 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:34.226918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerStarted","Data":"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"}
Jul 31 15:04:34.227121 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:34.226973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerStarted","Data":"cbe8000e8111bcfaec51cd262411f14ad1d23482826f1a40b2bf5b2903f44566"}
Jul 31 15:04:38.246969 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:38.246933    2566 generic.go:356] "Generic (PLEG): container finished" podID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerID="7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39" exitCode=0
Jul 31 15:04:38.247351 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:38.247004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerDied","Data":"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"}
Jul 31 15:04:39.253165 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:39.253131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerStarted","Data":"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"}
Jul 31 15:04:39.253533 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:39.253196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerStarted","Data":"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"}
Jul 31 15:04:39.253533 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:39.253386    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:39.273969 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:39.273924    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podStartSLOduration=2.589620225 podStartE2EDuration="6.27391103s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.684290805s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:04:39.273238197 +0000 UTC m=+1021.985212728" watchObservedRunningTime="2026-07-31 15:04:39.27391103 +0000 UTC m=+1021.985885558"
Jul 31 15:04:40.257479 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:40.257448    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:40.258648 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:40.258611    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:04:41.262254 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:41.262211    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:04:46.267014 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:46.266981    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:04:46.267582 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:46.267554    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:04:56.268358 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:04:56.268311    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:03.482382 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.482347    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:05:03.486547 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.486520    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.489507 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.489486    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-126e1-kube-rbac-proxy-sar-config\""
Jul 31 15:05:03.489605 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.489546    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-126e1-predictor-serving-cert\""
Jul 31 15:05:03.496733 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.496711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:05:03.580176 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.580141    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqx7l\" (UniqueName: \"kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.580176 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.580176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.580366 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.580198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.680895 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.680864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.681067 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.680904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.681067 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.681011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqx7l\" (UniqueName: \"kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.681067 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:05:03.681040    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-serving-cert: secret "message-dumper-raw-126e1-predictor-serving-cert" not found
Jul 31 15:05:03.681471 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:05:03.681114    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls podName:de5dfd10-2a4c-4f0b-b716-1500162f62bf nodeName:}" failed. No retries permitted until 2026-07-31 15:05:04.181094164 +0000 UTC m=+1046.893068675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls") pod "message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" (UID: "de5dfd10-2a4c-4f0b-b716-1500162f62bf") : secret "message-dumper-raw-126e1-predictor-serving-cert" not found
Jul 31 15:05:03.681584 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.681568    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:03.690343 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:03.690322    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqx7l\" (UniqueName: \"kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:04.185101 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:04.185066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:04.187352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:04.187331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") pod \"message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") " pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:04.397737 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:04.397702    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:04.528726 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:04.528701    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:05:04.530901 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:05:04.530869    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5dfd10_2a4c_4f0b_b716_1500162f62bf.slice/crio-dc12c6eac3411e796e6d45f92eef9496fbe202756f7be9aa6bb1d607929ab058 WatchSource:0}: Error finding container dc12c6eac3411e796e6d45f92eef9496fbe202756f7be9aa6bb1d607929ab058: Status 404 returned error can't find the container with id dc12c6eac3411e796e6d45f92eef9496fbe202756f7be9aa6bb1d607929ab058
Jul 31 15:05:04.532842 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:04.532794    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 15:05:05.364851 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:05.364796    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerStarted","Data":"dc12c6eac3411e796e6d45f92eef9496fbe202756f7be9aa6bb1d607929ab058"}
Jul 31 15:05:06.267710 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.267668    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:06.371450 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.371416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerStarted","Data":"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"}
Jul 31 15:05:06.371619 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.371502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerStarted","Data":"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"}
Jul 31 15:05:06.371671 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.371642    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:06.371709 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.371692    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:06.373468 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.373448    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:06.390669 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:06.390627    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" podStartSLOduration=2.327591582 podStartE2EDuration="3.3906132s" totalImagesPullingTime="1.063021618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:05:06.388143408 +0000 UTC m=+1049.100117939" watchObservedRunningTime="2026-07-31 15:05:06.3906132 +0000 UTC m=+1049.102587788"
Jul 31 15:05:13.384731 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:13.384699    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:05:16.267714 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:16.267672    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:26.268018 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:26.267980    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:36.268084 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:36.267998    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:46.267552 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:46.267503    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 31 15:05:56.268994 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:05:56.268962    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:06:36.491114 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:36.491081    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:06:36.491729 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:36.491358    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container" containerID="cri-o://a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b" gracePeriod=30
Jul 31 15:06:36.491729 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:36.491367    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kube-rbac-proxy" containerID="cri-o://bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50" gracePeriod=30
Jul 31 15:06:36.786746 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:36.786657    2566 generic.go:356] "Generic (PLEG): container finished" podID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerID="bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50" exitCode=2
Jul 31 15:06:36.786746 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:36.786704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerDied","Data":"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"}
Jul 31 15:06:41.136941 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.136910    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:06:41.212756 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.212727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location\") pod \"e91aefa0-8a19-4a75-b685-57626fd328d8\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") "
Jul 31 15:06:41.212942 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.212765    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert\") pod \"e91aefa0-8a19-4a75-b685-57626fd328d8\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") "
Jul 31 15:06:41.212942 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.212796    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config\") pod \"e91aefa0-8a19-4a75-b685-57626fd328d8\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") "
Jul 31 15:06:41.212942 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.212837    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb9hb\" (UniqueName: \"kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb\") pod \"e91aefa0-8a19-4a75-b685-57626fd328d8\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") "
Jul 31 15:06:41.212942 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.212876    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls\") pod \"e91aefa0-8a19-4a75-b685-57626fd328d8\" (UID: \"e91aefa0-8a19-4a75-b685-57626fd328d8\") "
Jul 31 15:06:41.213152 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.213122    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e91aefa0-8a19-4a75-b685-57626fd328d8" (UID: "e91aefa0-8a19-4a75-b685-57626fd328d8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:06:41.213193 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.213165    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e91aefa0-8a19-4a75-b685-57626fd328d8" (UID: "e91aefa0-8a19-4a75-b685-57626fd328d8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:06:41.213236 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.213168    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e91aefa0-8a19-4a75-b685-57626fd328d8" (UID: "e91aefa0-8a19-4a75-b685-57626fd328d8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:06:41.214851 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.214826    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e91aefa0-8a19-4a75-b685-57626fd328d8" (UID: "e91aefa0-8a19-4a75-b685-57626fd328d8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:06:41.215055 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.215032    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb" (OuterVolumeSpecName: "kube-api-access-gb9hb") pod "e91aefa0-8a19-4a75-b685-57626fd328d8" (UID: "e91aefa0-8a19-4a75-b685-57626fd328d8"). InnerVolumeSpecName "kube-api-access-gb9hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:06:41.314028 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.314000    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e91aefa0-8a19-4a75-b685-57626fd328d8-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:06:41.314028 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.314025    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:06:41.314202 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.314038    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:06:41.314202 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.314051    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb9hb\" (UniqueName: \"kubernetes.io/projected/e91aefa0-8a19-4a75-b685-57626fd328d8-kube-api-access-gb9hb\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:06:41.314202 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.314061    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e91aefa0-8a19-4a75-b685-57626fd328d8-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:06:41.810893 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.810857    2566 generic.go:356] "Generic (PLEG): container finished" podID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerID="a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b" exitCode=0
Jul 31 15:06:41.811190 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.810942    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"
Jul 31 15:06:41.814693 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.814664    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerDied","Data":"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"}
Jul 31 15:06:41.814828 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.814725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5" event={"ID":"e91aefa0-8a19-4a75-b685-57626fd328d8","Type":"ContainerDied","Data":"cbe8000e8111bcfaec51cd262411f14ad1d23482826f1a40b2bf5b2903f44566"}
Jul 31 15:06:41.814828 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.814798    2566 scope.go:117] "RemoveContainer" containerID="bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"
Jul 31 15:06:41.823799 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.823782    2566 scope.go:117] "RemoveContainer" containerID="a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"
Jul 31 15:06:41.831383 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.831366    2566 scope.go:117] "RemoveContainer" containerID="7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"
Jul 31 15:06:41.838600 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.838584    2566 scope.go:117] "RemoveContainer" containerID="bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"
Jul 31 15:06:41.838862 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:06:41.838839    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50\": container with ID starting with bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50 not found: ID does not exist" containerID="bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"
Jul 31 15:06:41.838951 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.838875    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50"} err="failed to get container status \"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50\": rpc error: code = NotFound desc = could not find container \"bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50\": container with ID starting with bac58378d9831014180ca8cfb8f7d1cdf1f0fe757c7baa33a4b6b863bc79cb50 not found: ID does not exist"
Jul 31 15:06:41.838951 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.838909    2566 scope.go:117] "RemoveContainer" containerID="a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"
Jul 31 15:06:41.839150 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:06:41.839131    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b\": container with ID starting with a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b not found: ID does not exist" containerID="a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"
Jul 31 15:06:41.839203 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.839159    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b"} err="failed to get container status \"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b\": rpc error: code = NotFound desc = could not find container \"a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b\": container with ID starting with a07f087d2ecef83a03cd78b31a01d0860132cc24945ac008c0674f83ff7e670b not found: ID does not exist"
Jul 31 15:06:41.839203 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.839180    2566 scope.go:117] "RemoveContainer" containerID="7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"
Jul 31 15:06:41.839444 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:06:41.839425    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39\": container with ID starting with 7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39 not found: ID does not exist" containerID="7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"
Jul 31 15:06:41.839593 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.839448    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39"} err="failed to get container status \"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39\": rpc error: code = NotFound desc = could not find container \"7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39\": container with ID starting with 7b9c2dacf6f4b1a9a36a35b27ce7b70075764a93836bf2fa4a65e4c82b140c39 not found: ID does not exist"
Jul 31 15:06:41.842491 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.842471    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:06:41.849092 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:41.849071    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-74ffcf-predictor-74d5dd6f74-m4gd5"]
Jul 31 15:06:43.814952 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:06:43.814917    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" path="/var/lib/kubelet/pods/e91aefa0-8a19-4a75-b685-57626fd328d8/volumes"
Jul 31 15:07:08.567245 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.567161    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f_de5dfd10-2a4c-4f0b-b716-1500162f62bf/kserve-container/0.log"
Jul 31 15:07:08.861245 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861166    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:07:08.861585 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861571    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kube-rbac-proxy"
Jul 31 15:07:08.861630 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861587    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kube-rbac-proxy"
Jul 31 15:07:08.861630 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861600    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="storage-initializer"
Jul 31 15:07:08.861630 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861606    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="storage-initializer"
Jul 31 15:07:08.861630 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861614    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container"
Jul 31 15:07:08.861630 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861623    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container"
Jul 31 15:07:08.861795 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861689    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kserve-container"
Jul 31 15:07:08.861795 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.861699    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e91aefa0-8a19-4a75-b685-57626fd328d8" containerName="kube-rbac-proxy"
Jul 31 15:07:08.865009 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.864990    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.868283 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.868257    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-97536-kube-rbac-proxy-sar-config\""
Jul 31 15:07:08.868283 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.868280    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 15:07:08.868616 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.868595    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-97536-predictor-serving-cert\""
Jul 31 15:07:08.868752 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.868738    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 15:07:08.876482 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.876457    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:07:08.947719 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.947687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfjs\" (UniqueName: \"kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.947931 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.947734    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.947931 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.947898    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.948055 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.947955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.948118 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.948062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:08.962993 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.962965    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:07:08.963220 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.963180    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kserve-container" containerID="cri-o://d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97" gracePeriod=30
Jul 31 15:07:08.963285 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:08.963207    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kube-rbac-proxy" containerID="cri-o://144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d" gracePeriod=30
Jul 31 15:07:09.048879 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.048839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049066 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.048889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049066 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.048930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049066 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.048971    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfjs\" (UniqueName: \"kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049066 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.049009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049297 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:07:09.049138    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-serving-cert: secret "isvc-sklearn-scale-raw-97536-predictor-serving-cert" not found
Jul 31 15:07:09.049297 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:07:09.049219    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls podName:94e127d2-98e1-464b-8038-3c5a6cc22cd9 nodeName:}" failed. No retries permitted until 2026-07-31 15:07:09.549197475 +0000 UTC m=+1172.261172002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls") pod "isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9") : secret "isvc-sklearn-scale-raw-97536-predictor-serving-cert" not found
Jul 31 15:07:09.049446 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.049424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049608 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.049587    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.049671 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.049619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.057652 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.057629    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfjs\" (UniqueName: \"kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.204616 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.204590    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:07:09.352899 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.352849    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") pod \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") "
Jul 31 15:07:09.353095 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.352934    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqx7l\" (UniqueName: \"kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l\") pod \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") "
Jul 31 15:07:09.353095 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.353009    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config\") pod \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\" (UID: \"de5dfd10-2a4c-4f0b-b716-1500162f62bf\") "
Jul 31 15:07:09.353412 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.353376    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "de5dfd10-2a4c-4f0b-b716-1500162f62bf" (UID: "de5dfd10-2a4c-4f0b-b716-1500162f62bf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:07:09.354946 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.354918    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "de5dfd10-2a4c-4f0b-b716-1500162f62bf" (UID: "de5dfd10-2a4c-4f0b-b716-1500162f62bf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:07:09.355040 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.354986    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l" (OuterVolumeSpecName: "kube-api-access-vqx7l") pod "de5dfd10-2a4c-4f0b-b716-1500162f62bf" (UID: "de5dfd10-2a4c-4f0b-b716-1500162f62bf"). InnerVolumeSpecName "kube-api-access-vqx7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:07:09.454063 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.454031    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqx7l\" (UniqueName: \"kubernetes.io/projected/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-api-access-vqx7l\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:07:09.454226 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.454074    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de5dfd10-2a4c-4f0b-b716-1500162f62bf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:07:09.454226 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.454085    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de5dfd10-2a4c-4f0b-b716-1500162f62bf-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:07:09.554663 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.554627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.557008 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.556986    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") pod \"isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.776226 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.776138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:09.907548 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.907520    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:07:09.909121 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:07:09.909094    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e127d2_98e1_464b_8038_3c5a6cc22cd9.slice/crio-5f1639f51f55f6fdeeb4311b2cdbb11806f1c8c29c4b68576555dd2f3d4ea289 WatchSource:0}: Error finding container 5f1639f51f55f6fdeeb4311b2cdbb11806f1c8c29c4b68576555dd2f3d4ea289: Status 404 returned error can't find the container with id 5f1639f51f55f6fdeeb4311b2cdbb11806f1c8c29c4b68576555dd2f3d4ea289
Jul 31 15:07:09.934574 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934548    2566 generic.go:356] "Generic (PLEG): container finished" podID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerID="144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d" exitCode=2
Jul 31 15:07:09.934574 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934570    2566 generic.go:356] "Generic (PLEG): container finished" podID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerID="d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97" exitCode=2
Jul 31 15:07:09.934720 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934594    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"
Jul 31 15:07:09.934720 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934626    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerDied","Data":"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"}
Jul 31 15:07:09.934720 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934687    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerDied","Data":"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"}
Jul 31 15:07:09.934880 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934718    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f" event={"ID":"de5dfd10-2a4c-4f0b-b716-1500162f62bf","Type":"ContainerDied","Data":"dc12c6eac3411e796e6d45f92eef9496fbe202756f7be9aa6bb1d607929ab058"}
Jul 31 15:07:09.934880 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.934753    2566 scope.go:117] "RemoveContainer" containerID="144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"
Jul 31 15:07:09.935867 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.935835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerStarted","Data":"5f1639f51f55f6fdeeb4311b2cdbb11806f1c8c29c4b68576555dd2f3d4ea289"}
Jul 31 15:07:09.950352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.950332    2566 scope.go:117] "RemoveContainer" containerID="d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"
Jul 31 15:07:09.962875 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.962828    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:07:09.964437 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.964414    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-126e1-predictor-7fb8f757df-lkd7f"]
Jul 31 15:07:09.966487 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.966330    2566 scope.go:117] "RemoveContainer" containerID="144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"
Jul 31 15:07:09.966639 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:07:09.966622    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d\": container with ID starting with 144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d not found: ID does not exist" containerID="144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"
Jul 31 15:07:09.966687 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.966650    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"} err="failed to get container status \"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d\": rpc error: code = NotFound desc = could not find container \"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d\": container with ID starting with 144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d not found: ID does not exist"
Jul 31 15:07:09.966687 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.966671    2566 scope.go:117] "RemoveContainer" containerID="d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"
Jul 31 15:07:09.966958 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:07:09.966939    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97\": container with ID starting with d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97 not found: ID does not exist" containerID="d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"
Jul 31 15:07:09.967023 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.966964    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"} err="failed to get container status \"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97\": rpc error: code = NotFound desc = could not find container \"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97\": container with ID starting with d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97 not found: ID does not exist"
Jul 31 15:07:09.967023 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.966977    2566 scope.go:117] "RemoveContainer" containerID="144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"
Jul 31 15:07:09.967224 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.967200    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d"} err="failed to get container status \"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d\": rpc error: code = NotFound desc = could not find container \"144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d\": container with ID starting with 144308e2f917a24ef81b6d3c4d28ba73700ecd66c9cb2a4d1f79d725b88e339d not found: ID does not exist"
Jul 31 15:07:09.967270 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.967225    2566 scope.go:117] "RemoveContainer" containerID="d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"
Jul 31 15:07:09.967491 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:09.967467    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97"} err="failed to get container status \"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97\": rpc error: code = NotFound desc = could not find container \"d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97\": container with ID starting with d2d7f424e233a10170dd2cb4cbf9f7c0baa15fc37b45133a6a3ac30e243dff97 not found: ID does not exist"
Jul 31 15:07:10.942074 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:10.942037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerStarted","Data":"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"}
Jul 31 15:07:11.814274 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:11.814233    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" path="/var/lib/kubelet/pods/de5dfd10-2a4c-4f0b-b716-1500162f62bf/volumes"
Jul 31 15:07:13.956279 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:13.956246    2566 generic.go:356] "Generic (PLEG): container finished" podID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerID="9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf" exitCode=0
Jul 31 15:07:13.956684 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:13.956316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerDied","Data":"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"}
Jul 31 15:07:14.963758 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.963726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerStarted","Data":"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"}
Jul 31 15:07:14.964169 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.963787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerStarted","Data":"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"}
Jul 31 15:07:14.964169 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.964087    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:14.964169 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.964149    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:14.965668 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.965639    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:07:14.985302 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:14.985253    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podStartSLOduration=3.067195368 podStartE2EDuration="6.985237218s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.91804185s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:07:14.984411838 +0000 UTC m=+1177.696386381" watchObservedRunningTime="2026-07-31 15:07:14.985237218 +0000 UTC m=+1177.697211748"
Jul 31 15:07:15.968167 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:15.968125    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:07:20.973042 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:20.973015    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:07:20.973525 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:20.973461    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:07:30.974053 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:30.974013    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:07:37.757386 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:37.757354    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:07:37.760622 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:37.760598    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:07:37.761356 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:37.761338    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:07:37.764606 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:37.764590    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:07:40.974010 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:40.973973    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:07:50.973822 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:07:50.973772    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:00.974201 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:00.974163    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:10.973542 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:10.973498    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:20.974269 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:20.974232    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:30.973481 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:30.973386    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:32.267099 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267064    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:08:32.267467 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267454    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kube-rbac-proxy"
Jul 31 15:08:32.267512 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267468    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kube-rbac-proxy"
Jul 31 15:08:32.267512 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267477    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kserve-container"
Jul 31 15:08:32.267512 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267483    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kserve-container"
Jul 31 15:08:32.267614 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267536    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kserve-container"
Jul 31 15:08:32.267614 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.267544    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5dfd10-2a4c-4f0b-b716-1500162f62bf" containerName="kube-rbac-proxy"
Jul 31 15:08:32.270875 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.270855    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.274703 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.274680    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-runtime-b10fc-predictor-serving-cert\""
Jul 31 15:08:32.274960 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.274941    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-runtime-b10fc-kube-rbac-proxy-sar-config\""
Jul 31 15:08:32.308678 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.308654    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:08:32.341999 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.341973    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.342099 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.342003    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.342099 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.342039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.342179 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.342139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.342224 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.342177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h6dr\" (UniqueName: \"kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443529 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.443498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443644 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.443538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h6dr\" (UniqueName: \"kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443644 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.443564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443644 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.443581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443644 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.443617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.443886 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:08:32.443768    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-serving-cert: secret "raw-sklearn-runtime-b10fc-predictor-serving-cert" not found
Jul 31 15:08:32.443886 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:08:32.443858    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls podName:b641d31e-8292-47ca-aa6f-98a03479a4d6 nodeName:}" failed. No retries permitted until 2026-07-31 15:08:32.94383573 +0000 UTC m=+1255.655810240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls") pod "raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6") : secret "raw-sklearn-runtime-b10fc-predictor-serving-cert" not found
Jul 31 15:08:32.444032 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.444014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.444177 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.444159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.444214 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.444178    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.456398 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.456380    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h6dr\" (UniqueName: \"kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.947901 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.947870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:32.950204 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:32.950185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") pod \"raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:33.181490 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:33.181453    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:33.316388 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:33.316365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:08:33.318343 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:08:33.318319    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb641d31e_8292_47ca_aa6f_98a03479a4d6.slice/crio-43438acace41aaf1ad2d11516040ff727ad72e0e0a46a3ed4dd2ce4997f4a0e8 WatchSource:0}: Error finding container 43438acace41aaf1ad2d11516040ff727ad72e0e0a46a3ed4dd2ce4997f4a0e8: Status 404 returned error can't find the container with id 43438acace41aaf1ad2d11516040ff727ad72e0e0a46a3ed4dd2ce4997f4a0e8
Jul 31 15:08:34.317223 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:34.317186    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerStarted","Data":"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"}
Jul 31 15:08:34.317616 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:34.317244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerStarted","Data":"43438acace41aaf1ad2d11516040ff727ad72e0e0a46a3ed4dd2ce4997f4a0e8"}
Jul 31 15:08:38.335609 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:38.335571    2566 generic.go:356] "Generic (PLEG): container finished" podID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerID="fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f" exitCode=0
Jul 31 15:08:38.336068 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:38.335647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerDied","Data":"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"}
Jul 31 15:08:39.341979 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:39.341947    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerStarted","Data":"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"}
Jul 31 15:08:39.342360 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:39.342013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerStarted","Data":"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"}
Jul 31 15:08:39.342360 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:39.342222    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:39.365443 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:39.365378    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podStartSLOduration=3.416199129 podStartE2EDuration="7.365357571s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.949158442s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:08:39.364351362 +0000 UTC m=+1262.076325891" watchObservedRunningTime="2026-07-31 15:08:39.365357571 +0000 UTC m=+1262.077332102"
Jul 31 15:08:39.808871 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:39.808826    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:40.346722 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:40.346693    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:40.348034 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:40.347993    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:08:41.351438 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:41.351399    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:08:46.356766 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:46.356736    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:08:46.357315 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:46.357290    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:08:49.809075 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:49.809031    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:08:56.357872 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:56.357831    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:08:59.808728 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:08:59.808687    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:06.357782 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:06.357743    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:09:09.808848 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:09.808773    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:16.357522 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:16.357476    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:09:19.809118 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:19.809078    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:26.357457 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:26.357420    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:09:29.808920 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:29.808873    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:36.358100 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:36.358060    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:09:39.809174 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:39.809131    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:46.357299 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:46.357258    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:09:47.815638 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:47.815613    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:09:49.005528 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:49.005495    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:09:49.663590 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:49.663545    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" containerID="cri-o://d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698" gracePeriod=30
Jul 31 15:09:49.663784 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:49.663575    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy" containerID="cri-o://8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f" gracePeriod=30
Jul 31 15:09:50.670477 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:50.670441    2566 generic.go:356] "Generic (PLEG): container finished" podID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerID="8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f" exitCode=2
Jul 31 15:09:50.670863 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:50.670512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerDied","Data":"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"}
Jul 31 15:09:50.969142 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:50.969047    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 31 15:09:55.968383 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:55.968341    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 31 15:09:56.358978 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:56.358897    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:09:57.813491 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:57.813453    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 31 15:09:59.215368 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.215343    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:09:59.285831 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.285783    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") pod \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") "
Jul 31 15:09:59.286013 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.285859    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhfjs\" (UniqueName: \"kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs\") pod \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") "
Jul 31 15:09:59.286013 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.285892    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location\") pod \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") "
Jul 31 15:09:59.286013 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.285951    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config\") pod \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") "
Jul 31 15:09:59.286013 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.285973    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert\") pod \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\" (UID: \"94e127d2-98e1-464b-8038-3c5a6cc22cd9\") "
Jul 31 15:09:59.286262 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.286237    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "94e127d2-98e1-464b-8038-3c5a6cc22cd9" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:09:59.286481 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.286448    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "94e127d2-98e1-464b-8038-3c5a6cc22cd9" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:09:59.286481 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.286476    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "94e127d2-98e1-464b-8038-3c5a6cc22cd9" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:09:59.287884 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.287864    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "94e127d2-98e1-464b-8038-3c5a6cc22cd9" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:09:59.287988 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.287971    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs" (OuterVolumeSpecName: "kube-api-access-fhfjs") pod "94e127d2-98e1-464b-8038-3c5a6cc22cd9" (UID: "94e127d2-98e1-464b-8038-3c5a6cc22cd9"). InnerVolumeSpecName "kube-api-access-fhfjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:09:59.386882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.386794    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:09:59.386882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.386832    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/94e127d2-98e1-464b-8038-3c5a6cc22cd9-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:09:59.386882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.386843    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94e127d2-98e1-464b-8038-3c5a6cc22cd9-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:09:59.386882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.386852    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhfjs\" (UniqueName: \"kubernetes.io/projected/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kube-api-access-fhfjs\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:09:59.386882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.386861    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/94e127d2-98e1-464b-8038-3c5a6cc22cd9-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:09:59.712792 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.712759    2566 generic.go:356] "Generic (PLEG): container finished" podID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerID="d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698" exitCode=0
Jul 31 15:09:59.712979 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.712836    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"
Jul 31 15:09:59.712979 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.712847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerDied","Data":"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"}
Jul 31 15:09:59.712979 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.712915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj" event={"ID":"94e127d2-98e1-464b-8038-3c5a6cc22cd9","Type":"ContainerDied","Data":"5f1639f51f55f6fdeeb4311b2cdbb11806f1c8c29c4b68576555dd2f3d4ea289"}
Jul 31 15:09:59.712979 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.712942    2566 scope.go:117] "RemoveContainer" containerID="8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"
Jul 31 15:09:59.722216 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.722199    2566 scope.go:117] "RemoveContainer" containerID="d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"
Jul 31 15:09:59.729849 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.729829    2566 scope.go:117] "RemoveContainer" containerID="9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"
Jul 31 15:09:59.738253 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738233    2566 scope.go:117] "RemoveContainer" containerID="8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"
Jul 31 15:09:59.738370 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738348    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:09:59.738499 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:09:59.738481    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f\": container with ID starting with 8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f not found: ID does not exist" containerID="8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"
Jul 31 15:09:59.738547 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738513    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f"} err="failed to get container status \"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f\": rpc error: code = NotFound desc = could not find container \"8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f\": container with ID starting with 8a7df86b7c597322fa7a9514657b56ac3f5e6cbc2544144c07d675927960267f not found: ID does not exist"
Jul 31 15:09:59.738547 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738534    2566 scope.go:117] "RemoveContainer" containerID="d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"
Jul 31 15:09:59.738766 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:09:59.738749    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698\": container with ID starting with d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698 not found: ID does not exist" containerID="d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"
Jul 31 15:09:59.738833 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738771    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698"} err="failed to get container status \"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698\": rpc error: code = NotFound desc = could not find container \"d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698\": container with ID starting with d03e0114a50544dbaad0649fedbc7261e985838e882b4adae5e0b83f39ee2698 not found: ID does not exist"
Jul 31 15:09:59.738833 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.738787    2566 scope.go:117] "RemoveContainer" containerID="9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"
Jul 31 15:09:59.739048 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:09:59.739030    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf\": container with ID starting with 9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf not found: ID does not exist" containerID="9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"
Jul 31 15:09:59.739048 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.739049    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf"} err="failed to get container status \"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf\": rpc error: code = NotFound desc = could not find container \"9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf\": container with ID starting with 9d8d29591b80f5c1b9dac12e53764ed0633bd35e0ad60c8b2a8473d936a45dbf not found: ID does not exist"
Jul 31 15:09:59.741701 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.741679    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-97536-predictor-6c695549d5-ww4rj"]
Jul 31 15:09:59.814884 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:09:59.814855    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" path="/var/lib/kubelet/pods/94e127d2-98e1-464b-8038-3c5a6cc22cd9/volumes"
Jul 31 15:10:02.330780 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:02.330699    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:10:02.331208 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:02.330992    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" containerID="cri-o://2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47" gracePeriod=30
Jul 31 15:10:02.331208 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:02.331039    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kube-rbac-proxy" containerID="cri-o://a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23" gracePeriod=30
Jul 31 15:10:02.728976 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:02.728945    2566 generic.go:356] "Generic (PLEG): container finished" podID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerID="a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23" exitCode=2
Jul 31 15:10:02.729121 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:02.728990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerDied","Data":"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"}
Jul 31 15:10:06.351746 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:06.351701    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Jul 31 15:10:06.357224 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:06.357189    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 31 15:10:06.963925 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:06.963901    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:10:07.057716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.057629    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config\") pod \"b641d31e-8292-47ca-aa6f-98a03479a4d6\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") "
Jul 31 15:10:07.057716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.057669    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h6dr\" (UniqueName: \"kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr\") pod \"b641d31e-8292-47ca-aa6f-98a03479a4d6\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") "
Jul 31 15:10:07.057716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.057705    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location\") pod \"b641d31e-8292-47ca-aa6f-98a03479a4d6\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") "
Jul 31 15:10:07.058015 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.057787    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert\") pod \"b641d31e-8292-47ca-aa6f-98a03479a4d6\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") "
Jul 31 15:10:07.058015 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.057858    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") pod \"b641d31e-8292-47ca-aa6f-98a03479a4d6\" (UID: \"b641d31e-8292-47ca-aa6f-98a03479a4d6\") "
Jul 31 15:10:07.058178 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.058148    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b641d31e-8292-47ca-aa6f-98a03479a4d6" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:10:07.058178 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.058165    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b641d31e-8292-47ca-aa6f-98a03479a4d6" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:10:07.058286 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.058150    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b641d31e-8292-47ca-aa6f-98a03479a4d6" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:10:07.059793 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.059771    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b641d31e-8292-47ca-aa6f-98a03479a4d6" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:10:07.059890 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.059863    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr" (OuterVolumeSpecName: "kube-api-access-2h6dr") pod "b641d31e-8292-47ca-aa6f-98a03479a4d6" (UID: "b641d31e-8292-47ca-aa6f-98a03479a4d6"). InnerVolumeSpecName "kube-api-access-2h6dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:10:07.159256 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.159211    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:10:07.159256 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.159249    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h6dr\" (UniqueName: \"kubernetes.io/projected/b641d31e-8292-47ca-aa6f-98a03479a4d6-kube-api-access-2h6dr\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:10:07.159256 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.159260    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b641d31e-8292-47ca-aa6f-98a03479a4d6-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:10:07.159256 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.159271    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b641d31e-8292-47ca-aa6f-98a03479a4d6-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:10:07.159513 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.159280    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b641d31e-8292-47ca-aa6f-98a03479a4d6-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:10:07.752989 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.752955    2566 generic.go:356] "Generic (PLEG): container finished" podID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerID="2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47" exitCode=0
Jul 31 15:10:07.752989 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.752994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerDied","Data":"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"}
Jul 31 15:10:07.753502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.753009    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"
Jul 31 15:10:07.753502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.753055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r" event={"ID":"b641d31e-8292-47ca-aa6f-98a03479a4d6","Type":"ContainerDied","Data":"43438acace41aaf1ad2d11516040ff727ad72e0e0a46a3ed4dd2ce4997f4a0e8"}
Jul 31 15:10:07.753502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.753093    2566 scope.go:117] "RemoveContainer" containerID="a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"
Jul 31 15:10:07.761714 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.761690    2566 scope.go:117] "RemoveContainer" containerID="2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"
Jul 31 15:10:07.768948 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.768932    2566 scope.go:117] "RemoveContainer" containerID="fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"
Jul 31 15:10:07.776631 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.776612    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:10:07.776949 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.776927    2566 scope.go:117] "RemoveContainer" containerID="a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"
Jul 31 15:10:07.777200 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:10:07.777172    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23\": container with ID starting with a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23 not found: ID does not exist" containerID="a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"
Jul 31 15:10:07.777278 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.777203    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23"} err="failed to get container status \"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23\": rpc error: code = NotFound desc = could not find container \"a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23\": container with ID starting with a4b150e5c457c0b445493cadaa0b764508d930f904e85a8e83f88d3ecdbdda23 not found: ID does not exist"
Jul 31 15:10:07.777278 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.777219    2566 scope.go:117] "RemoveContainer" containerID="2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"
Jul 31 15:10:07.777463 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:10:07.777448    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47\": container with ID starting with 2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47 not found: ID does not exist" containerID="2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"
Jul 31 15:10:07.777507 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.777470    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47"} err="failed to get container status \"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47\": rpc error: code = NotFound desc = could not find container \"2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47\": container with ID starting with 2a14b88d5f71b32c5c788ca6bd9b2a4fd7fc4b94accefa63f7f5d06943b81f47 not found: ID does not exist"
Jul 31 15:10:07.777507 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.777485    2566 scope.go:117] "RemoveContainer" containerID="fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"
Jul 31 15:10:07.777743 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:10:07.777726    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f\": container with ID starting with fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f not found: ID does not exist" containerID="fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"
Jul 31 15:10:07.777788 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.777748    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f"} err="failed to get container status \"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f\": rpc error: code = NotFound desc = could not find container \"fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f\": container with ID starting with fafa234b5127dcd0b1718b6ae0d20a9d30a4a29098a41ddda789b0de00f7549f not found: ID does not exist"
Jul 31 15:10:07.782245 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.782227    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-b10fc-predictor-6ff74c8c75-hjp6r"]
Jul 31 15:10:07.813856 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:10:07.813834    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" path="/var/lib/kubelet/pods/b641d31e-8292-47ca-aa6f-98a03479a4d6/volumes"
Jul 31 15:11:29.490238 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490202    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490568    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kube-rbac-proxy"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490580    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kube-rbac-proxy"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490595    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490600    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490613    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490619    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490624    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490629    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490642    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="storage-initializer"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490648    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="storage-initializer"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490659    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="storage-initializer"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490664    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="storage-initializer"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490719    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490727    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kserve-container"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490732    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e127d2-98e1-464b-8038-3c5a6cc22cd9" containerName="kube-rbac-proxy"
Jul 31 15:11:29.490735 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.490740    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b641d31e-8292-47ca-aa6f-98a03479a4d6" containerName="kube-rbac-proxy"
Jul 31 15:11:29.493907 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.493877    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.496678 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.496655    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 15:11:29.496834 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.496661    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 15:11:29.496834 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.496730    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-predictor-serving-cert\""
Jul 31 15:11:29.496834 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.496745    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 15:11:29.496834 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.496776    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Jul 31 15:11:29.497596 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.497579    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 15:11:29.497645 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.497583    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mhdbw\""
Jul 31 15:11:29.507387 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.507365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:11:29.524841 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.524818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.524945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.524864    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.524945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.524883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.525028 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.524955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.525028 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.524988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frlm5\" (UniqueName: \"kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.625882 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.625852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626045 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.625904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626045 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.625932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626045 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.625984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626045 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.626020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frlm5\" (UniqueName: \"kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626373 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.626353    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626661 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.626644    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.626702 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.626679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.628355 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.628337    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.635520 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.635492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frlm5\" (UniqueName: \"kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5\") pod \"isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.804256 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.804169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:29.936012 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:29.935990    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:11:29.937789 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:11:29.937760    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3db776f_6505_4979_b4f3_3efe61a72f1a.slice/crio-b1950953413eb9b83111cee78ac33c2f7ef511b8af1167e237e5a1d3e775bb6f WatchSource:0}: Error finding container b1950953413eb9b83111cee78ac33c2f7ef511b8af1167e237e5a1d3e775bb6f: Status 404 returned error can't find the container with id b1950953413eb9b83111cee78ac33c2f7ef511b8af1167e237e5a1d3e775bb6f
Jul 31 15:11:30.112181 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:30.112088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerStarted","Data":"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"}
Jul 31 15:11:30.112181 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:30.112152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerStarted","Data":"b1950953413eb9b83111cee78ac33c2f7ef511b8af1167e237e5a1d3e775bb6f"}
Jul 31 15:11:34.131627 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:34.131538    2566 generic.go:356] "Generic (PLEG): container finished" podID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerID="e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205" exitCode=0
Jul 31 15:11:34.131627 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:34.131596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerDied","Data":"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"}
Jul 31 15:11:35.138695 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:35.138657    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerStarted","Data":"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"}
Jul 31 15:11:35.139147 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:35.138721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerStarted","Data":"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"}
Jul 31 15:11:35.139147 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:35.138931    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:35.159952 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:35.159890    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podStartSLOduration=2.331493796 podStartE2EDuration="6.159870477s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.828376681s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:11:35.157417187 +0000 UTC m=+1437.869391713" watchObservedRunningTime="2026-07-31 15:11:35.159870477 +0000 UTC m=+1437.871845010"
Jul 31 15:11:36.143530 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:36.143499    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:36.144841 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:36.144797    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:11:37.147518 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:37.147483    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:11:42.151866 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:42.151833    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:11:42.152313 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:42.152285    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:11:52.152642 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:11:52.152601    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:02.153251 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:02.153211    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:12.152586 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:12.152543    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:22.153230 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:22.153191    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:32.152689 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:32.152644    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:37.788419 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:37.788393    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:12:37.789545 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:37.789526    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:12:37.791593 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:37.791572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:12:37.792699 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:37.792681    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:12:42.152542 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:42.152495    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 31 15:12:52.153966 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:12:52.153937    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:13:04.579650 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.579613    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:13:04.580092 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.579903    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container" containerID="cri-o://98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0" gracePeriod=30
Jul 31 15:13:04.580092 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.579957    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kube-rbac-proxy" containerID="cri-o://d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3" gracePeriod=30
Jul 31 15:13:04.659172 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.659147    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:13:04.663038 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.663021    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.665559 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.665536    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Jul 31 15:13:04.674621 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.674598    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:13:04.823753 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.823726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvvkb\" (UniqueName: \"kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.823945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.823756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.823945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.823794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.823945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.823866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.823945 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.823928    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925030 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvvkb\" (UniqueName: \"kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925173 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925173 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925173 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925173 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925589 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925914 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.925967 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.925893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.927572 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.927551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.934199 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.934177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvvkb\" (UniqueName: \"kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb\") pod \"isvc-canary-promote-v2-predictor-b87c95ccc-t72nh\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:04.973707 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:04.973683    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:05.110535 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:05.110512    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:13:05.112120 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:13:05.112096    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54d7fd6_866b_426c_95e6_4099daa08567.slice/crio-f7a5b392ac7534dc15142cb645782f6dded47ae571217191f56b53edffec90ed WatchSource:0}: Error finding container f7a5b392ac7534dc15142cb645782f6dded47ae571217191f56b53edffec90ed: Status 404 returned error can't find the container with id f7a5b392ac7534dc15142cb645782f6dded47ae571217191f56b53edffec90ed
Jul 31 15:13:05.568610 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:05.568568    2566 generic.go:356] "Generic (PLEG): container finished" podID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerID="d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3" exitCode=2
Jul 31 15:13:05.568787 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:05.568643    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerDied","Data":"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"}
Jul 31 15:13:05.570155 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:05.570132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerStarted","Data":"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"}
Jul 31 15:13:05.570273 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:05.570182    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerStarted","Data":"f7a5b392ac7534dc15142cb645782f6dded47ae571217191f56b53edffec90ed"}
Jul 31 15:13:07.148613 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:07.148570    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 31 15:13:08.935047 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:08.935018    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:13:09.062335 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062306    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frlm5\" (UniqueName: \"kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5\") pod \"c3db776f-6505-4979-b4f3-3efe61a72f1a\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") "
Jul 31 15:13:09.062455 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062358    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location\") pod \"c3db776f-6505-4979-b4f3-3efe61a72f1a\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") "
Jul 31 15:13:09.062455 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062378    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config\") pod \"c3db776f-6505-4979-b4f3-3efe61a72f1a\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") "
Jul 31 15:13:09.062455 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls\") pod \"c3db776f-6505-4979-b4f3-3efe61a72f1a\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") "
Jul 31 15:13:09.062455 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062452    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert\") pod \"c3db776f-6505-4979-b4f3-3efe61a72f1a\" (UID: \"c3db776f-6505-4979-b4f3-3efe61a72f1a\") "
Jul 31 15:13:09.062756 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062730    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c3db776f-6505-4979-b4f3-3efe61a72f1a" (UID: "c3db776f-6505-4979-b4f3-3efe61a72f1a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:13:09.062848 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062756    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c3db776f-6505-4979-b4f3-3efe61a72f1a" (UID: "c3db776f-6505-4979-b4f3-3efe61a72f1a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:13:09.062911 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.062867    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c3db776f-6505-4979-b4f3-3efe61a72f1a" (UID: "c3db776f-6505-4979-b4f3-3efe61a72f1a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:13:09.064496 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.064474    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5" (OuterVolumeSpecName: "kube-api-access-frlm5") pod "c3db776f-6505-4979-b4f3-3efe61a72f1a" (UID: "c3db776f-6505-4979-b4f3-3efe61a72f1a"). InnerVolumeSpecName "kube-api-access-frlm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:13:09.064575 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.064520    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c3db776f-6505-4979-b4f3-3efe61a72f1a" (UID: "c3db776f-6505-4979-b4f3-3efe61a72f1a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:13:09.163878 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.163857    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:13:09.163878 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.163878    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frlm5\" (UniqueName: \"kubernetes.io/projected/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-api-access-frlm5\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:13:09.164002 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.163888    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c3db776f-6505-4979-b4f3-3efe61a72f1a-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:13:09.164002 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.163897    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3db776f-6505-4979-b4f3-3efe61a72f1a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:13:09.164002 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.163906    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3db776f-6505-4979-b4f3-3efe61a72f1a-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:13:09.593613 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.593579    2566 generic.go:356] "Generic (PLEG): container finished" podID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerID="98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0" exitCode=0
Jul 31 15:13:09.593842 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.593661    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"
Jul 31 15:13:09.593842 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.593658    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerDied","Data":"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"}
Jul 31 15:13:09.593963 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.593853    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b" event={"ID":"c3db776f-6505-4979-b4f3-3efe61a72f1a","Type":"ContainerDied","Data":"b1950953413eb9b83111cee78ac33c2f7ef511b8af1167e237e5a1d3e775bb6f"}
Jul 31 15:13:09.593963 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.593899    2566 scope.go:117] "RemoveContainer" containerID="d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"
Jul 31 15:13:09.595151 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.595128    2566 generic.go:356] "Generic (PLEG): container finished" podID="c54d7fd6-866b-426c-95e6-4099daa08567" containerID="05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27" exitCode=0
Jul 31 15:13:09.595248 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.595185    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerDied","Data":"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"}
Jul 31 15:13:09.603383 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.603362    2566 scope.go:117] "RemoveContainer" containerID="98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"
Jul 31 15:13:09.611586 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.611559    2566 scope.go:117] "RemoveContainer" containerID="e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"
Jul 31 15:13:09.620454 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.620417    2566 scope.go:117] "RemoveContainer" containerID="d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"
Jul 31 15:13:09.620695 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:13:09.620677    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3\": container with ID starting with d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3 not found: ID does not exist" containerID="d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"
Jul 31 15:13:09.620751 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.620703    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3"} err="failed to get container status \"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3\": rpc error: code = NotFound desc = could not find container \"d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3\": container with ID starting with d82a33510f91371a0714b18632c4cb4606f50328ec293bcbb41087cfe73362c3 not found: ID does not exist"
Jul 31 15:13:09.620751 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.620719    2566 scope.go:117] "RemoveContainer" containerID="98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"
Jul 31 15:13:09.620988 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:13:09.620964    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0\": container with ID starting with 98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0 not found: ID does not exist" containerID="98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"
Jul 31 15:13:09.621067 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.620998    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0"} err="failed to get container status \"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0\": rpc error: code = NotFound desc = could not find container \"98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0\": container with ID starting with 98946c4d50b6b361f47a15692db12d9ef98c75dfb5ab54541334994ed6101fa0 not found: ID does not exist"
Jul 31 15:13:09.621067 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.621019    2566 scope.go:117] "RemoveContainer" containerID="e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"
Jul 31 15:13:09.621253 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:13:09.621239    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205\": container with ID starting with e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205 not found: ID does not exist" containerID="e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"
Jul 31 15:13:09.621304 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.621258    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205"} err="failed to get container status \"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205\": rpc error: code = NotFound desc = could not find container \"e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205\": container with ID starting with e6bd6cf86451d0e71b5844eee34ccd260eb03cad1726d9a287b884cfb794c205 not found: ID does not exist"
Jul 31 15:13:09.634975 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.634953    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:13:09.638753 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.638734    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-7cd9bb4b6b-4xp8b"]
Jul 31 15:13:09.815598 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:09.815571    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" path="/var/lib/kubelet/pods/c3db776f-6505-4979-b4f3-3efe61a72f1a/volumes"
Jul 31 15:13:10.602876 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:10.602846    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerStarted","Data":"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"}
Jul 31 15:13:10.603243 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:10.602899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerStarted","Data":"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"}
Jul 31 15:13:10.603243 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:10.603103    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:10.623286 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:10.623244    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podStartSLOduration=2.745302374 podStartE2EDuration="6.623230859s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.877928485s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:13:10.622131512 +0000 UTC m=+1533.334106071" watchObservedRunningTime="2026-07-31 15:13:10.623230859 +0000 UTC m=+1533.335205389"
Jul 31 15:13:11.608339 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:11.608307    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:11.609784 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:11.609758    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:12.617484 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:12.617442    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:17.622636 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:17.622610    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:13:17.623137 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:17.623108    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:27.623112 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:27.623070    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:37.624058 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:37.624015    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:47.623618 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:47.623583    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:13:57.623554 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:13:57.623513    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:14:07.624034 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:07.623989    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:14:17.623238 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:17.623198    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:14:19.809437 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:19.809380    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 31 15:14:29.813780 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:29.813752    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:14:34.854853 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:34.854736    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:14:34.855236 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:34.855013    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container" containerID="cri-o://ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12" gracePeriod=30
Jul 31 15:14:34.855236 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:34.855047    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kube-rbac-proxy" containerID="cri-o://c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561" gracePeriod=30
Jul 31 15:14:34.993544 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:34.993514    2566 generic.go:356] "Generic (PLEG): container finished" podID="c54d7fd6-866b-426c-95e6-4099daa08567" containerID="c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561" exitCode=2
Jul 31 15:14:34.993668 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:34.993591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerDied","Data":"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"}
Jul 31 15:14:37.618652 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:37.618606    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused"
Jul 31 15:14:39.307307 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.307281    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:14:39.413661 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.413579    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls\") pod \"c54d7fd6-866b-426c-95e6-4099daa08567\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") "
Jul 31 15:14:39.413661 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.413623    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvvkb\" (UniqueName: \"kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb\") pod \"c54d7fd6-866b-426c-95e6-4099daa08567\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") "
Jul 31 15:14:39.413661 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.413643    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config\") pod \"c54d7fd6-866b-426c-95e6-4099daa08567\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") "
Jul 31 15:14:39.413947 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.413678    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location\") pod \"c54d7fd6-866b-426c-95e6-4099daa08567\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") "
Jul 31 15:14:39.413947 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.413694    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert\") pod \"c54d7fd6-866b-426c-95e6-4099daa08567\" (UID: \"c54d7fd6-866b-426c-95e6-4099daa08567\") "
Jul 31 15:14:39.414059 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.414025    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c54d7fd6-866b-426c-95e6-4099daa08567" (UID: "c54d7fd6-866b-426c-95e6-4099daa08567"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 15:14:39.414123 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.414089    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c54d7fd6-866b-426c-95e6-4099daa08567" (UID: "c54d7fd6-866b-426c-95e6-4099daa08567"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:14:39.414214 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.414190    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c54d7fd6-866b-426c-95e6-4099daa08567" (UID: "c54d7fd6-866b-426c-95e6-4099daa08567"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:14:39.415787 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.415761    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c54d7fd6-866b-426c-95e6-4099daa08567" (UID: "c54d7fd6-866b-426c-95e6-4099daa08567"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:14:39.415964 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.415941    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb" (OuterVolumeSpecName: "kube-api-access-gvvkb") pod "c54d7fd6-866b-426c-95e6-4099daa08567" (UID: "c54d7fd6-866b-426c-95e6-4099daa08567"). InnerVolumeSpecName "kube-api-access-gvvkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:14:39.515120 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.515087    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c54d7fd6-866b-426c-95e6-4099daa08567-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:14:39.515120 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.515115    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvvkb\" (UniqueName: \"kubernetes.io/projected/c54d7fd6-866b-426c-95e6-4099daa08567-kube-api-access-gvvkb\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:14:39.515352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.515129    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:14:39.515352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.515143    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c54d7fd6-866b-426c-95e6-4099daa08567-kserve-provision-location\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:14:39.515352 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:39.515155    2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c54d7fd6-866b-426c-95e6-4099daa08567-cabundle-cert\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:14:40.020502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.020467    2566 generic.go:356] "Generic (PLEG): container finished" podID="c54d7fd6-866b-426c-95e6-4099daa08567" containerID="ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12" exitCode=0
Jul 31 15:14:40.020701 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.020524    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"
Jul 31 15:14:40.020701 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.020551    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerDied","Data":"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"}
Jul 31 15:14:40.020701 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.020622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh" event={"ID":"c54d7fd6-866b-426c-95e6-4099daa08567","Type":"ContainerDied","Data":"f7a5b392ac7534dc15142cb645782f6dded47ae571217191f56b53edffec90ed"}
Jul 31 15:14:40.020701 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.020650    2566 scope.go:117] "RemoveContainer" containerID="c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"
Jul 31 15:14:40.029140 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.029111    2566 scope.go:117] "RemoveContainer" containerID="ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"
Jul 31 15:14:40.036299 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.036281    2566 scope.go:117] "RemoveContainer" containerID="05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"
Jul 31 15:14:40.042133 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.042109    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:14:40.044502 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.044487    2566 scope.go:117] "RemoveContainer" containerID="c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"
Jul 31 15:14:40.044732 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:14:40.044712    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561\": container with ID starting with c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561 not found: ID does not exist" containerID="c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"
Jul 31 15:14:40.044793 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.044740    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561"} err="failed to get container status \"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561\": rpc error: code = NotFound desc = could not find container \"c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561\": container with ID starting with c396f89dcb151267eb596f53e88c63ec185b40761be5ffa2c8aa23ad8817a561 not found: ID does not exist"
Jul 31 15:14:40.044793 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.044759    2566 scope.go:117] "RemoveContainer" containerID="ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"
Jul 31 15:14:40.045017 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:14:40.045003    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12\": container with ID starting with ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12 not found: ID does not exist" containerID="ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"
Jul 31 15:14:40.045056 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.045021    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12"} err="failed to get container status \"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12\": rpc error: code = NotFound desc = could not find container \"ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12\": container with ID starting with ec46e0b3dddb92c9020509008afd8a7296f822d9f1eb0420f7e7fb094fe10c12 not found: ID does not exist"
Jul 31 15:14:40.045056 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.045033    2566 scope.go:117] "RemoveContainer" containerID="05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"
Jul 31 15:14:40.045242 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:14:40.045225    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27\": container with ID starting with 05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27 not found: ID does not exist" containerID="05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"
Jul 31 15:14:40.045278 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.045248    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27"} err="failed to get container status \"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27\": rpc error: code = NotFound desc = could not find container \"05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27\": container with ID starting with 05457e361c0f86d27fd77c75362b0a47807f569e0da5e27102438534d98a9d27 not found: ID does not exist"
Jul 31 15:14:40.047964 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:40.047946    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b87c95ccc-t72nh"]
Jul 31 15:14:41.814230 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:41.814194    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" path="/var/lib/kubelet/pods/c54d7fd6-866b-426c-95e6-4099daa08567/volumes"
Jul 31 15:14:48.729631 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.729595    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:14:48.730170 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730147    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kube-rbac-proxy"
Jul 31 15:14:48.730334 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730307    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kube-rbac-proxy"
Jul 31 15:14:48.730334 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730333    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kube-rbac-proxy"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730342    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kube-rbac-proxy"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730367    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730375    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730388    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="storage-initializer"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730397    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="storage-initializer"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730417    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730428    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730449    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="storage-initializer"
Jul 31 15:14:48.730476 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730455    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="storage-initializer"
Jul 31 15:14:48.730745 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730536    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kserve-container"
Jul 31 15:14:48.730745 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730544    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kserve-container"
Jul 31 15:14:48.730745 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730552    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3db776f-6505-4979-b4f3-3efe61a72f1a" containerName="kube-rbac-proxy"
Jul 31 15:14:48.730745 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.730558    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54d7fd6-866b-426c-95e6-4099daa08567" containerName="kube-rbac-proxy"
Jul 31 15:14:48.734954 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.734936    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:48.738500 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.738480    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-kube-rbac-proxy-sar-config\""
Jul 31 15:14:48.739026 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.739005    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 15:14:48.739103 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.739018    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 15:14:48.740005 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.739988    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mhdbw\""
Jul 31 15:14:48.740091 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.739994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-predictor-serving-cert\""
Jul 31 15:14:48.756915 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.756895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:14:48.902180 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.902137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czln8\" (UniqueName: \"kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:48.902348 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.902194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:48.902348 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:48.902317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.003009 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.002929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.003009 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.002972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czln8\" (UniqueName: \"kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.003009 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.003002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.003601 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.003579    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.005336 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.005315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.011510 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.011483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czln8\" (UniqueName: \"kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-lwqqg\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.045339 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.045315    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:49.177182 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:49.177157    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:14:49.179154 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:14:49.179126    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5efe9bf_9130_484b_bb1d_3041832dd882.slice/crio-4847dc175efa66374f2ce6c8397b08977b84c754e99abef5c2b4b696ab1b430b WatchSource:0}: Error finding container 4847dc175efa66374f2ce6c8397b08977b84c754e99abef5c2b4b696ab1b430b: Status 404 returned error can't find the container with id 4847dc175efa66374f2ce6c8397b08977b84c754e99abef5c2b4b696ab1b430b
Jul 31 15:14:50.067716 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.067678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerStarted","Data":"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"}
Jul 31 15:14:50.068156 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.067747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerStarted","Data":"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"}
Jul 31 15:14:50.068156 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.067838    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:50.068156 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.067887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerStarted","Data":"4847dc175efa66374f2ce6c8397b08977b84c754e99abef5c2b4b696ab1b430b"}
Jul 31 15:14:50.068156 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.067912    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:50.069948 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.069928    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:14:50.086776 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:50.086725    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" podStartSLOduration=2.086714766 podStartE2EDuration="2.086714766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 15:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:14:50.086213675 +0000 UTC m=+1632.798188205" watchObservedRunningTime="2026-07-31 15:14:50.086714766 +0000 UTC m=+1632.798689297"
Jul 31 15:14:57.081636 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:14:57.081607    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:16:33.759870 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:33.759783    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-cipn-predictor-9774bf55d-lwqqg_d5efe9bf-9130-484b-bb1d-3041832dd882/kserve-container/0.log"
Jul 31 15:16:34.052669 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.052585    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:16:34.052891 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.052865    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kserve-container" containerID="cri-o://b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc" gracePeriod=30
Jul 31 15:16:34.052994 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.052921    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kube-rbac-proxy" containerID="cri-o://30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854" gracePeriod=30
Jul 31 15:16:34.297984 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.297956    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:16:34.387427 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.387363    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls\") pod \"d5efe9bf-9130-484b-bb1d-3041832dd882\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") "
Jul 31 15:16:34.387427 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.387401    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config\") pod \"d5efe9bf-9130-484b-bb1d-3041832dd882\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") "
Jul 31 15:16:34.387571 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.387446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czln8\" (UniqueName: \"kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8\") pod \"d5efe9bf-9130-484b-bb1d-3041832dd882\" (UID: \"d5efe9bf-9130-484b-bb1d-3041832dd882\") "
Jul 31 15:16:34.387851 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.387792    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d5efe9bf-9130-484b-bb1d-3041832dd882" (UID: "d5efe9bf-9130-484b-bb1d-3041832dd882"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 15:16:34.389440 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.389411    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8" (OuterVolumeSpecName: "kube-api-access-czln8") pod "d5efe9bf-9130-484b-bb1d-3041832dd882" (UID: "d5efe9bf-9130-484b-bb1d-3041832dd882"). InnerVolumeSpecName "kube-api-access-czln8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 15:16:34.389518 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.389433    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d5efe9bf-9130-484b-bb1d-3041832dd882" (UID: "d5efe9bf-9130-484b-bb1d-3041832dd882"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 15:16:34.488093 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.488073    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czln8\" (UniqueName: \"kubernetes.io/projected/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-api-access-czln8\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:16:34.488182 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.488095    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5efe9bf-9130-484b-bb1d-3041832dd882-proxy-tls\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:16:34.488182 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.488105    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5efe9bf-9130-484b-bb1d-3041832dd882-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-204.ec2.internal\" DevicePath \"\""
Jul 31 15:16:34.526351 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526327    2566 generic.go:356] "Generic (PLEG): container finished" podID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerID="30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854" exitCode=2
Jul 31 15:16:34.526351 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526350    2566 generic.go:356] "Generic (PLEG): container finished" podID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerID="b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc" exitCode=2
Jul 31 15:16:34.526514 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526389    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"
Jul 31 15:16:34.526514 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerDied","Data":"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"}
Jul 31 15:16:34.526514 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerDied","Data":"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"}
Jul 31 15:16:34.526514 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg" event={"ID":"d5efe9bf-9130-484b-bb1d-3041832dd882","Type":"ContainerDied","Data":"4847dc175efa66374f2ce6c8397b08977b84c754e99abef5c2b4b696ab1b430b"}
Jul 31 15:16:34.526698 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.526517    2566 scope.go:117] "RemoveContainer" containerID="30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"
Jul 31 15:16:34.543176 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.543160    2566 scope.go:117] "RemoveContainer" containerID="b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"
Jul 31 15:16:34.550572 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.550540    2566 scope.go:117] "RemoveContainer" containerID="30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"
Jul 31 15:16:34.550899 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:16:34.550876    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854\": container with ID starting with 30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854 not found: ID does not exist" containerID="30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"
Jul 31 15:16:34.550989 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.550910    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"} err="failed to get container status \"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854\": rpc error: code = NotFound desc = could not find container \"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854\": container with ID starting with 30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854 not found: ID does not exist"
Jul 31 15:16:34.550989 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.550936    2566 scope.go:117] "RemoveContainer" containerID="b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"
Jul 31 15:16:34.551266 ip-10-0-132-204 kubenswrapper[2566]: E0731 15:16:34.551234    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc\": container with ID starting with b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc not found: ID does not exist" containerID="b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"
Jul 31 15:16:34.551360 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.551273    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"} err="failed to get container status \"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc\": rpc error: code = NotFound desc = could not find container \"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc\": container with ID starting with b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc not found: ID does not exist"
Jul 31 15:16:34.551360 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.551289    2566 scope.go:117] "RemoveContainer" containerID="30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"
Jul 31 15:16:34.551707 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.551683    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854"} err="failed to get container status \"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854\": rpc error: code = NotFound desc = could not find container \"30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854\": container with ID starting with 30e12530186fb2389fa8a953e9d056d3e1b7c6820733a2738835e6aa56fed854 not found: ID does not exist"
Jul 31 15:16:34.551707 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.551707    2566 scope.go:117] "RemoveContainer" containerID="b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"
Jul 31 15:16:34.551970 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.551949    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc"} err="failed to get container status \"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc\": rpc error: code = NotFound desc = could not find container \"b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc\": container with ID starting with b72c83077ec822eb5907e898a286d8d5f0d8fee89daefafbfe627827f35929cc not found: ID does not exist"
Jul 31 15:16:34.553177 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.553158    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:16:34.556989 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:34.556971    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-lwqqg"]
Jul 31 15:16:35.813802 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:16:35.813772    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" path="/var/lib/kubelet/pods/d5efe9bf-9130-484b-bb1d-3041832dd882/volumes"
Jul 31 15:17:10.995066 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:10.995027    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vdqg9_9bc66d4e-9a34-4922-b231-cf1c4793428a/global-pull-secret-syncer/0.log"
Jul 31 15:17:11.098439 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:11.098407    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vfpmp_bfdcf735-c1a2-407b-9ce7-f32a4b0780dd/konnectivity-agent/0.log"
Jul 31 15:17:11.159541 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:11.159498    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-204.ec2.internal_346e694eb0cadd3e9407624267497090/haproxy/0.log"
Jul 31 15:17:15.890457 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:15.890366    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/alertmanager/0.log"
Jul 31 15:17:15.917662 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:15.917638    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/config-reloader/0.log"
Jul 31 15:17:15.943694 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:15.943663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/kube-rbac-proxy-web/0.log"
Jul 31 15:17:15.969789 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:15.969767    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/kube-rbac-proxy/0.log"
Jul 31 15:17:15.997698 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:15.997659    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/kube-rbac-proxy-metric/0.log"
Jul 31 15:17:16.040654 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.040629    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/prom-label-proxy/0.log"
Jul 31 15:17:16.068130 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.068078    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0aae37fa-d9cb-426f-a6f8-c328ca269408/init-config-reloader/0.log"
Jul 31 15:17:16.106932 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.106908    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-7pkb4_131917c6-74ad-4b9c-8fbe-f75a9e57193c/cluster-monitoring-operator/0.log"
Jul 31 15:17:16.131017 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.130998    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-v8pwh_43b700ef-dc72-404b-9bb0-dc8da84f512b/kube-state-metrics/0.log"
Jul 31 15:17:16.152916 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.152867    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-v8pwh_43b700ef-dc72-404b-9bb0-dc8da84f512b/kube-rbac-proxy-main/0.log"
Jul 31 15:17:16.175217 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.175197    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-v8pwh_43b700ef-dc72-404b-9bb0-dc8da84f512b/kube-rbac-proxy-self/0.log"
Jul 31 15:17:16.245576 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.245552    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-m256w_f7936908-70fc-48fb-8e2e-ae9828f58b79/monitoring-plugin/0.log"
Jul 31 15:17:16.471477 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.471449    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlvvq_a31b6a53-e665-475a-a54f-6330a32ff800/node-exporter/0.log"
Jul 31 15:17:16.501249 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.501220    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlvvq_a31b6a53-e665-475a-a54f-6330a32ff800/kube-rbac-proxy/0.log"
Jul 31 15:17:16.531526 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.531499    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlvvq_a31b6a53-e665-475a-a54f-6330a32ff800/init-textfile/0.log"
Jul 31 15:17:16.665846 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.665802    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-lfqld_614252f9-ade6-4829-99d5-ca836e7223ad/kube-rbac-proxy-main/0.log"
Jul 31 15:17:16.695212 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.695182    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-lfqld_614252f9-ade6-4829-99d5-ca836e7223ad/kube-rbac-proxy-self/0.log"
Jul 31 15:17:16.719919 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:16.719894    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-lfqld_614252f9-ade6-4829-99d5-ca836e7223ad/openshift-state-metrics/0.log"
Jul 31 15:17:17.136852 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:17.136800    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567664c68f-69cks_b647eca0-5731-4660-9b64-9810c27be9ac/telemeter-client/0.log"
Jul 31 15:17:17.166271 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:17.166243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567664c68f-69cks_b647eca0-5731-4660-9b64-9810c27be9ac/reload/0.log"
Jul 31 15:17:17.200317 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:17.200286    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567664c68f-69cks_b647eca0-5731-4660-9b64-9810c27be9ac/kube-rbac-proxy/0.log"
Jul 31 15:17:19.092553 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.092489    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/2.log"
Jul 31 15:17:19.096271 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.096251    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-n2v68_e00e5e8b-4f02-406d-a8aa-3966ed31acae/console-operator/3.log"
Jul 31 15:17:19.583813 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.583770    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-xdnp5_d510c2f1-c5ab-4eb5-962f-33d3eaaa1b99/download-server/0.log"
Jul 31 15:17:19.584120 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584100    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"]
Jul 31 15:17:19.584477 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584464    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kube-rbac-proxy"
Jul 31 15:17:19.584523 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584479    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kube-rbac-proxy"
Jul 31 15:17:19.584523 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584490    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kserve-container"
Jul 31 15:17:19.584523 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584495    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kserve-container"
Jul 31 15:17:19.584615 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584550    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kserve-container"
Jul 31 15:17:19.584615 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.584560    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5efe9bf-9130-484b-bb1d-3041832dd882" containerName="kube-rbac-proxy"
Jul 31 15:17:19.587508 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.587488    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.590459 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.590433    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zszp\"/\"kube-root-ca.crt\""
Jul 31 15:17:19.591532 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.591513    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zszp\"/\"openshift-service-ca.crt\""
Jul 31 15:17:19.591633 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.591542    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4zszp\"/\"default-dockercfg-7tg6t\""
Jul 31 15:17:19.599964 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.599940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"]
Jul 31 15:17:19.647596 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.647561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-podres\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.647759 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.647622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-proc\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.647759 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.647656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-lib-modules\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.647759 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.647674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-sys\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.647759 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.647698    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5kxx\" (UniqueName: \"kubernetes.io/projected/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-kube-api-access-m5kxx\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748496 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748457    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5kxx\" (UniqueName: \"kubernetes.io/projected/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-kube-api-access-m5kxx\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-podres\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-proc\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-lib-modules\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-sys\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748668    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-podres\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748912 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-proc\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748912 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748722    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-sys\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.748912 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.748772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-lib-modules\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.757142 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.757116    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5kxx\" (UniqueName: \"kubernetes.io/projected/3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73-kube-api-access-m5kxx\") pod \"perf-node-gather-daemonset-td6ns\" (UID: \"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73\") " pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:19.897033 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:19.896936    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:20.030712 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.030682    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"]
Jul 31 15:17:20.032167 ip-10-0-132-204 kubenswrapper[2566]: W0731 15:17:20.032128    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f1be4a8_55eb_40bb_aa5e_3f48b56c4d73.slice/crio-e4c4770f75b4db21b1a691a29c415171d9513365ca8efb4abc317a06a102d076 WatchSource:0}: Error finding container e4c4770f75b4db21b1a691a29c415171d9513365ca8efb4abc317a06a102d076: Status 404 returned error can't find the container with id e4c4770f75b4db21b1a691a29c415171d9513365ca8efb4abc317a06a102d076
Jul 31 15:17:20.192227 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.192201    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-9rhl5_aec941e5-d6ec-4f49-a23c-25291523162f/volume-data-source-validator/0.log"
Jul 31 15:17:20.727105 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.727069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns" event={"ID":"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73","Type":"ContainerStarted","Data":"20079b086d0e7a0ea8b8a50b43801f9559bc653225d1d614044d741f1c776705"}
Jul 31 15:17:20.727281 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.727123    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns" event={"ID":"3f1be4a8-55eb-40bb-aa5e-3f48b56c4d73","Type":"ContainerStarted","Data":"e4c4770f75b4db21b1a691a29c415171d9513365ca8efb4abc317a06a102d076"}
Jul 31 15:17:20.727281 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.727159    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:20.745175 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:20.745126    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns" podStartSLOduration=1.745106017 podStartE2EDuration="1.745106017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 15:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 15:17:20.74437581 +0000 UTC m=+1783.456350373" watchObservedRunningTime="2026-07-31 15:17:20.745106017 +0000 UTC m=+1783.457080545"
Jul 31 15:17:21.212581 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:21.212554    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2xrq_baca90d9-9cb5-43f1-a5b5-47826d395d31/dns/0.log"
Jul 31 15:17:21.235326 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:21.235305    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2xrq_baca90d9-9cb5-43f1-a5b5-47826d395d31/kube-rbac-proxy/0.log"
Jul 31 15:17:21.287399 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:21.287371    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krg8d_f634de43-ec15-469d-95e8-932e660ca0ab/dns-node-resolver/0.log"
Jul 31 15:17:21.772873 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:21.772842    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bhw7_fc7cce3c-f9e2-450a-ac7f-4f3cbd157f27/node-ca/0.log"
Jul 31 15:17:22.566143 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:22.566115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-587cb6667d-857jd_2c39bfbf-b3d4-4aa9-8c0e-3c5a6ffac179/router/0.log"
Jul 31 15:17:22.935483 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:22.935457    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5qq4_dbd836fa-93a0-4c34-a332-c3606ef36615/serve-healthcheck-canary/0.log"
Jul 31 15:17:23.422144 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:23.422119    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-lwjqb_a6802a36-92bb-4e6a-a8ad-d86898381e2c/insights-operator/0.log"
Jul 31 15:17:23.422313 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:23.422199    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-lwjqb_a6802a36-92bb-4e6a-a8ad-d86898381e2c/insights-operator/1.log"
Jul 31 15:17:23.529946 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:23.529916    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8g29_09f1d198-7f7a-4cea-b8fd-1806ac828fde/kube-rbac-proxy/0.log"
Jul 31 15:17:23.557448 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:23.557412    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8g29_09f1d198-7f7a-4cea-b8fd-1806ac828fde/exporter/0.log"
Jul 31 15:17:23.581905 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:23.581882    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8g29_09f1d198-7f7a-4cea-b8fd-1806ac828fde/extractor/0.log"
Jul 31 15:17:26.009527 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:26.005168    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-fcc9dd4db-8n8t2_bfa5a128-b8f3-45fc-b0d0-c907c9ced0d5/manager/0.log"
Jul 31 15:17:26.071616 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:26.071590    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-4k9t6_6954f627-91a3-42b7-b4c0-7acef0c1af7a/server/0.log"
Jul 31 15:17:26.741486 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:26.741458    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4zszp/perf-node-gather-daemonset-td6ns"
Jul 31 15:17:31.435068 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:31.435036    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-bxpvh_b3aad374-423d-4a82-8a0d-e2947ecc6168/kube-storage-version-migrator-operator/1.log"
Jul 31 15:17:31.435820 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:31.435791    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-bxpvh_b3aad374-423d-4a82-8a0d-e2947ecc6168/kube-storage-version-migrator-operator/0.log"
Jul 31 15:17:32.616895 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.616867    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/kube-multus-additional-cni-plugins/0.log"
Jul 31 15:17:32.647358 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.647327    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/egress-router-binary-copy/0.log"
Jul 31 15:17:32.676677 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.676651    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/cni-plugins/0.log"
Jul 31 15:17:32.704891 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.704865    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/bond-cni-plugin/0.log"
Jul 31 15:17:32.736414 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.736387    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/routeoverride-cni/0.log"
Jul 31 15:17:32.760621 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.760588    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/whereabouts-cni-bincopy/0.log"
Jul 31 15:17:32.789625 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:32.789596    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9nkf_5304b752-64c8-4148-9ba4-ff4c6c5633ce/whereabouts-cni/0.log"
Jul 31 15:17:33.112445 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:33.112411    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ndp56_3506513f-4c3f-404c-9a31-ff2f07b34287/kube-multus/0.log"
Jul 31 15:17:33.207984 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:33.207951    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n8x7h_1b29e6e9-f8e0-451e-b658-cbf0ed5ab089/network-metrics-daemon/0.log"
Jul 31 15:17:33.234712 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:33.234685    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n8x7h_1b29e6e9-f8e0-451e-b658-cbf0ed5ab089/kube-rbac-proxy/0.log"
Jul 31 15:17:34.618583 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.618555    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-controller/0.log"
Jul 31 15:17:34.641933 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.641907    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/0.log"
Jul 31 15:17:34.642089 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.641940    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovn-acl-logging/1.log"
Jul 31 15:17:34.663687 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.663654    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/kube-rbac-proxy-node/0.log"
Jul 31 15:17:34.688098 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.688072    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/kube-rbac-proxy-ovn-metrics/0.log"
Jul 31 15:17:34.709639 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.709611    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/northd/0.log"
Jul 31 15:17:34.732987 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.732962    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/nbdb/0.log"
Jul 31 15:17:34.755790 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.755765    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/sbdb/0.log"
Jul 31 15:17:34.795381 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:34.795351    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksbpc_6a96c46b-2413-49da-8c07-f5ce93e629c0/ovnkube-controller/0.log"
Jul 31 15:17:35.885770 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:35.885735    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7vzl5_4f3ebed1-1af2-4f49-b77e-14efe59b929c/network-check-target-container/0.log"
Jul 31 15:17:36.943955 ip-10-0-132-204 kubenswrapper[2566]: I0731 15:17:36.943926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lqtlz_69954056-b580-451d-b1f2-806a0c9ec9e3/iptables-alerter/0.log"