Jul 01 14:39:09.932646 ip-10-0-133-214 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 14:39:10.329249 ip-10-0-133-214 kubenswrapper[2541]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:10.329249 ip-10-0-133-214 kubenswrapper[2541]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 14:39:10.329249 ip-10-0-133-214 kubenswrapper[2541]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:10.329249 ip-10-0-133-214 kubenswrapper[2541]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 14:39:10.329249 ip-10-0-133-214 kubenswrapper[2541]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:10.331079 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.330992    2541 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 14:39:10.336107 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336090    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.336107 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336108    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336112    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336117    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336120    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336122    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336125    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336128    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336140    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336143    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336148    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336152    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336155    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336157    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336160    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336162    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336165    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336167    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336170    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336172    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.336172 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336175    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336178    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336181    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336184    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336187    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336189    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336192    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336194    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336196    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336199    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336202    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336204    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336206    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336209    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336211    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336214    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336217    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336219    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336223    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336226    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.336642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336229    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336231    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336236    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336239    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336242    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336245    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336247    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336250    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336253    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336255    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336258    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336260    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336264    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336267    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336269    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336272    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336275    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336277    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336280    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.337139 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336282    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336285    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336287    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336290    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336292    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336294    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336297    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336299    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336302    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336304    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336307    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336309    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336312    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336314    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336317    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336319    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336322    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336324    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336327    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336329    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.337606 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336331    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336334    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336336    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336339    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336342    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336345    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336348    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336351    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336353    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336356    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336358    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336362    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336366    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336368    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336371    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336374    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336377    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336379    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336382    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.338083 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.336398    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338132    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338139    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338142    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338146    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338149    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338152    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338155    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338157    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338160    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338162    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338165    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338167    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338170    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338172    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338175    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338177    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338179    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338182    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338184    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.338561 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338188    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338190    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338192    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338195    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338198    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338200    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338202    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338205    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338208    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338211    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338215    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338218    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338221    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338224    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338227    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338230    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338233    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338236    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338240    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.339099 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338243    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338246    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338248    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338251    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338253    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338256    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338258    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338261    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338263    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338266    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338268    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338270    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338273    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338275    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338278    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338280    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338283    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338285    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338288    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338290    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.339578 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338292    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338296    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338300    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338303    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338306    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338309    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338312    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338315    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338317    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338320    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338322    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338325    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338327    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338330    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338333    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338335    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338338    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338340    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338343    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.340045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338346    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338348    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338350    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338353    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338355    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338358    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338360    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338362    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338365    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338367    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338370    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338372    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338374    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338377    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338379    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338382    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338398    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338401    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338406    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338410    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.340531 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338413    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338416    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338797    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338802    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338804    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338807    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338811    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338813    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338816    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338818    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338821    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338825    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338829    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338832    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338835    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338837    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338840    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338842    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338845    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.341008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338847    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338850    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338852    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338862    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338865    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338868    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338871    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338874    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338876    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338879    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338881    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338884    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338887    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338889    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338892    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338894    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338897    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338899    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338902    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338904    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.341467 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338908    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338910    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338913    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338915    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338918    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338920    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338922    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338926    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338928    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338930    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338933    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338936    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338938    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338940    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338943    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338945    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338948    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338950    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338953    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338955    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.341952 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338957    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338960    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338962    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338965    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338967    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338970    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338973    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338977    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338979    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338982    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338985    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338988    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338991    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338994    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338996    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.338999    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339003    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339005    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339008    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339010    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.342441 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339012    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339015    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339017    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339020    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339023    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339026    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339029    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339031    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339034    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339036    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339039    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339041    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339043    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339046    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339048    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339050    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339053    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339055    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339057    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.342930 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339060    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339062    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339065    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339134    2541 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339144    2541 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339151    2541 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339155    2541 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339160    2541 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339163    2541 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339167    2541 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339172    2541 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339175    2541 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339177    2541 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339181    2541 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339184    2541 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339187    2541 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339190    2541 flags.go:64] FLAG: --cgroup-root=""
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339193    2541 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339196    2541 flags.go:64] FLAG: --client-ca-file=""
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339199    2541 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339202    2541 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339206    2541 flags.go:64] FLAG: --cluster-domain=""
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339208    2541 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339211    2541 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 14:39:10.343378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339214    2541 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339217    2541 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339221    2541 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339224    2541 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339226    2541 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339230    2541 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339232    2541 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339235    2541 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339238    2541 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339241    2541 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339243    2541 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339247    2541 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339250    2541 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339253    2541 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339255    2541 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339258    2541 flags.go:64] FLAG: --enable-server="true"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339262    2541 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339268    2541 flags.go:64] FLAG: --event-burst="100"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339271    2541 flags.go:64] FLAG: --event-qps="50"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339274    2541 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339277    2541 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339279    2541 flags.go:64] FLAG: --eviction-hard=""
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339284    2541 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339287    2541 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339290    2541 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 14:39:10.343992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339293    2541 flags.go:64] FLAG: --eviction-soft=""
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339295    2541 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339298    2541 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339301    2541 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339304    2541 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339307    2541 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339309    2541 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339312    2541 flags.go:64] FLAG: --feature-gates=""
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339315    2541 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339318    2541 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339321    2541 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339324    2541 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339327    2541 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339330    2541 flags.go:64] FLAG: --help="false"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339332    2541 flags.go:64] FLAG: --hostname-override="ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339335    2541 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339338    2541 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339341    2541 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339344    2541 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339347    2541 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339350    2541 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339353    2541 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339359    2541 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 14:39:10.344587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339362    2541 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339365    2541 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339368    2541 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339371    2541 flags.go:64] FLAG: --kube-reserved=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339374    2541 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339376    2541 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339382    2541 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339400    2541 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339403    2541 flags.go:64] FLAG: --lock-file=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339405    2541 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339408    2541 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339411    2541 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339416    2541 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339419    2541 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339422    2541 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339424    2541 flags.go:64] FLAG: --logging-format="text"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339427    2541 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339430    2541 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339433    2541 flags.go:64] FLAG: --manifest-url=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339436    2541 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339440    2541 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339443    2541 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339447    2541 flags.go:64] FLAG: --max-pods="110"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339449    2541 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339452    2541 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 14:39:10.345151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339455    2541 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339458    2541 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339461    2541 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339464    2541 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339467    2541 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339473    2541 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339477    2541 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339480    2541 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339483    2541 flags.go:64] FLAG: --pod-cidr=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339486    2541 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339492    2541 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339495    2541 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339497    2541 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339500    2541 flags.go:64] FLAG: --port="10250"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339505    2541 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339508    2541 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0154164603693dc3d"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339511    2541 flags.go:64] FLAG: --qos-reserved=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339513    2541 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339516    2541 flags.go:64] FLAG: --register-node="true"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339519    2541 flags.go:64] FLAG: --register-with-taints=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339523    2541 flags.go:64] FLAG: --registry-burst="10"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339526    2541 flags.go:64] FLAG: --registry-qps="5"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339528    2541 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339531    2541 flags.go:64] FLAG: --reserved-memory=""
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339535    2541 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 14:39:10.345783 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339537    2541 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339540    2541 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339543    2541 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339546    2541 flags.go:64] FLAG: --runonce="false"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339549    2541 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339552    2541 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339554    2541 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339557    2541 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339560    2541 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339563    2541 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339566    2541 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339568    2541 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339571    2541 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339575    2541 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339578    2541 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339581    2541 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339584    2541 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339587    2541 flags.go:64] FLAG: --system-cgroups=""
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339590    2541 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339595    2541 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339598    2541 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339602    2541 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339606    2541 flags.go:64] FLAG: --tls-min-version=""
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339609    2541 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339611    2541 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 14:39:10.346379 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339614    2541 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339617    2541 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339619    2541 flags.go:64] FLAG: --v="2"
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339623    2541 flags.go:64] FLAG: --version="false"
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339627    2541 flags.go:64] FLAG: --vmodule=""
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339630    2541 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.339633    2541 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339743    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339746    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339749    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339752    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339755    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339758    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339761    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339764    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339766    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339769    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339771    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339774    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339778    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.347047 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339785    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339788    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339792    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339794    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339797    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339800    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339803    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339806    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339812    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339814    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339817    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339820    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339822    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339825    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339827    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339830    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339832    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339835    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339837    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339840    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.347559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339842    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339845    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339847    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339850    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339852    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339855    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339857    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339860    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339862    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339865    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339868    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339870    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339874    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339877    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339880    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339882    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339885    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339887    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339890    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339893    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.348045 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339896    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339899    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339901    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339904    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339906    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339908    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339911    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339914    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339916    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339918    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339921    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339924    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339926    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339930    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339932    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339935    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339938    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339940    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339943    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.348548 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339945    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339947    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339950    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339952    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339955    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339958    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339961    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339965    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339968    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339971    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339974    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339976    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339979    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339984    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339986    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339989    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339992    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339994    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339996    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.339999    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.349003 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340001    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340004    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340006    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340008    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340011    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340013    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.340016    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.340021    2541 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.346615    2541 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.346628    2541 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.348646    2541 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.348654    2541 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348713    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348718    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348721    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.349508 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348724    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348726    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348729    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348732    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348735    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348737    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348740    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348743    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348745    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348748    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348750    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348754    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348756    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348758    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348761    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348764    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348767    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348769    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348773    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.349873 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348777    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348780    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348783    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348785    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348787    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348790    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348792    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348796    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348798    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348801    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348804    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348806    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348809    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348811    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348814    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348816    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348819    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348821    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348824    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348826    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.350356 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348828    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348831    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348833    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348835    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348838    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348840    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348843    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348845    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348847    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348850    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348852    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348855    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348857    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348860    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348862    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348865    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348867    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348870    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348872    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348874    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.350838 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348879    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348883    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348885    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348888    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348891    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348893    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348896    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348898    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348901    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348904    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348907    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348909    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348912    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348914    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348917    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348919    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348922    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348924    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348927    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348929    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.351318 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348931    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348934    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348936    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348939    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348941    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348943    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348946    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348949    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348952    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348955    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348958    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348961    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348963    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348966    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348968    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348970    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.351815 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.348973    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.348977    2541 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349089    2541 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349093    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349096    2541 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349099    2541 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349102    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349104    2541 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349107    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349110    2541 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349112    2541 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349115    2541 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349117    2541 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349120    2541 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:10.352191 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349122    2541 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349125    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349127    2541 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349130    2541 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349132    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349134    2541 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349137    2541 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349139    2541 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349142    2541 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349144    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349146    2541 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349149    2541 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349151    2541 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349154    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349156    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349159    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349165    2541 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349167    2541 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349170    2541 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:10.352552 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349172    2541 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349174    2541 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349177    2541 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349179    2541 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349181    2541 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349184    2541 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349186    2541 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349188    2541 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349191    2541 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349193    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349196    2541 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349198    2541 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349201    2541 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349203    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349205    2541 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349208    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349211    2541 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349213    2541 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349216    2541 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349218    2541 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:10.353008 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349220    2541 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349223    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349225    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349228    2541 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349230    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349232    2541 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349235    2541 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349237    2541 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349239    2541 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349242    2541 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349245    2541 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349247    2541 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349249    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349252    2541 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349254    2541 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349257    2541 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349259    2541 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349261    2541 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349264    2541 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349266    2541 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:10.353642 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349268    2541 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349271    2541 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349273    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349276    2541 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349278    2541 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349281    2541 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349283    2541 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349285    2541 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349288    2541 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349290    2541 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349294    2541 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349298    2541 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349302    2541 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349305    2541 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349308    2541 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349311    2541 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349314    2541 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349318    2541 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349321    2541 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:10.354137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349324    2541 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349329    2541 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349332    2541 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349334    2541 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349338    2541 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349340    2541 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349342    2541 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349345    2541 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:10.349347    2541 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.349352    2541 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.349475    2541 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.351518    2541 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.352899    2541 server.go:1019] "Starting client certificate rotation"
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.353004    2541 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 14:39:10.354614 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.353069    2541 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 14:39:10.355278 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.355264    2541 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 14:39:10.359164 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.359141    2541 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 14:39:10.370422 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.370404    2541 log.go:25] "Validated CRI v1 runtime API"
Jul 01 14:39:10.372440 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.372424    2541 log.go:25] "Validated CRI v1 image API"
Jul 01 14:39:10.373287 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.373274    2541 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 14:39:10.378050 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.378025    2541 fs.go:135] Filesystem UUIDs: map[5318ae8a-7bab-44ad-ae03-107cab79605c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 98547a0b-6863-464c-a05d-a3185fe5c61f:/dev/nvme0n1p4]
Jul 01 14:39:10.378124 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.378049    2541 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 14:39:10.383178 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.383073    2541 manager.go:216] Machine: {Timestamp:2026-07-01 14:39:10.381687983 +0000 UTC m=+0.336526040 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102115 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26539135785a8afb3df06ca05ecc58 SystemUUID:ec265391-3578-5a8a-fb3d-f06ca05ecc58 BootID:41ea6f95-b971-4d10-a62b-8bb4f5038d12 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:15:90:8b:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:15:90:8b:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:ed:62:dc:aa:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 14:39:10.383178 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.383174    2541 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 14:39:10.383280 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.383255    2541 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 14:39:10.386697 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.386671    2541 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 14:39:10.386829 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.386702    2541 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-214.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 14:39:10.387421 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.387411    2541 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 14:39:10.387457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.387424    2541 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 14:39:10.387457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.387431    2541 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 14:39:10.388183 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.388172    2541 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 14:39:10.388949 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.388940    2541 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 14:39:10.391204 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.391187    2541 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 14:39:10.391931 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.391920    2541 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 14:39:10.392046 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392037    2541 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 14:39:10.392081 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392066    2541 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 14:39:10.392081 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392077    2541 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 14:39:10.392156 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392090    2541 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 14:39:10.392156 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392103    2541 kubelet.go:398] "Adding apiserver pod source"
Jul 01 14:39:10.392156 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392112    2541 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 14:39:10.392485 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392469    2541 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 14:39:10.392540 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.392494    2541 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 14:39:10.394770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.394752    2541 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 14:39:10.396138 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396120    2541 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 14:39:10.396223 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396160    2541 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 14:39:10.396364 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396351    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 14:39:10.396447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396371    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 14:39:10.396447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396380    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 14:39:10.396447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396405    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 14:39:10.396447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396419    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 14:39:10.396447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396427    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 14:39:10.396983 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396972    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 14:39:10.397037 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396985    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 14:39:10.397037 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.396998    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 14:39:10.397037 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.397008    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 14:39:10.397037 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.397021    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 14:39:10.397037 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.397033    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 14:39:10.398794 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.398780    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 14:39:10.398863 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.398797    2541 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 14:39:10.403333 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.403316    2541 server.go:1268] "Started kubelet"
Jul 01 14:39:10.403439 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.403382    2541 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 14:39:10.403581 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.403539    2541 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 14:39:10.403629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.403602    2541 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 14:39:10.404147 ip-10-0-133-214 systemd[1]: Started Kubernetes Kubelet.
Jul 01 14:39:10.404687 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.404546    2541 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-214.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 14:39:10.404687 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.404577    2541 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-214.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 14:39:10.404877 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.404765    2541 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 14:39:10.404877 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.404800    2541 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 14:39:10.405840 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.405827    2541 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 14:39:10.408627 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.408606    2541 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 14:39:10.410981 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.410962    2541 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 14:39:10.410981 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.410970    2541 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 14:39:10.411115 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.411098    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.411591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411561    2541 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 14:39:10.411662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411597    2541 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 14:39:10.411662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411582    2541 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 14:39:10.411757 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411712    2541 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 14:39:10.411757 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411722    2541 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 14:39:10.412013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411960    2541 factory.go:55] Registering systemd factory
Jul 01 14:39:10.412013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.411973    2541 factory.go:223] Registration of the systemd container factory successfully
Jul 01 14:39:10.412288 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412254    2541 factory.go:153] Registering CRI-O factory
Jul 01 14:39:10.412288 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412266    2541 factory.go:223] Registration of the crio container factory successfully
Jul 01 14:39:10.412447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412310    2541 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 14:39:10.412447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412327    2541 factory.go:103] Registering Raw factory
Jul 01 14:39:10.412447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412338    2541 manager.go:1196] Started watching for new ooms in manager
Jul 01 14:39:10.412877 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.412865    2541 manager.go:349] Starting recovery of all containers
Jul 01 14:39:10.420485 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.420318    2541 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-214.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 01 14:39:10.420485 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.420337    2541 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 01 14:39:10.423482 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.422362    2541 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-214.ec2.internal.18be315693e1a9c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-214.ec2.internal,UID:ip-10-0-133-214.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-214.ec2.internal,},FirstTimestamp:2026-07-01 14:39:10.403283393 +0000 UTC m=+0.358121449,LastTimestamp:2026-07-01 14:39:10.403283393 +0000 UTC m=+0.358121449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-214.ec2.internal,}"
Jul 01 14:39:10.424720 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.424694    2541 manager.go:354] Recovery completed
Jul 01 14:39:10.430054 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.430040    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.431997 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.431972    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.432056 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432014    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.432056 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432024    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.432613 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432600    2541 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 14:39:10.432613 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432613    2541 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 14:39:10.432736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432632    2541 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 14:39:10.432736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.432602    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.433362 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433347    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.433441 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433376    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.433441 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433407    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.433971 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433959    2541 policy_none.go:49] "None policy: Start"
Jul 01 14:39:10.434035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433978    2541 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 14:39:10.434035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.433992    2541 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 14:39:10.435060 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.434966    2541 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-214.ec2.internal.18be31569597c9b6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-214.ec2.internal,UID:ip-10-0-133-214.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-214.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-214.ec2.internal,},FirstTimestamp:2026-07-01 14:39:10.431996342 +0000 UTC m=+0.386834399,LastTimestamp:2026-07-01 14:39:10.431996342 +0000 UTC m=+0.386834399,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-214.ec2.internal,}"
Jul 01 14:39:10.435151 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.435073    2541 policy_none.go:47] "Start"
Jul 01 14:39:10.445009 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.444935    2541 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-214.ec2.internal.18be3156959822c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-214.ec2.internal,UID:ip-10-0-133-214.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-214.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-214.ec2.internal,},FirstTimestamp:2026-07-01 14:39:10.432019144 +0000 UTC m=+0.386857204,LastTimestamp:2026-07-01 14:39:10.432019144 +0000 UTC m=+0.386857204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-214.ec2.internal,}"
Jul 01 14:39:10.455921 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.455832    2541 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-214.ec2.internal.18be315695984663  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-214.ec2.internal,UID:ip-10-0-133-214.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-214.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-214.ec2.internal,},FirstTimestamp:2026-07-01 14:39:10.432028259 +0000 UTC m=+0.386866316,LastTimestamp:2026-07-01 14:39:10.432028259 +0000 UTC m=+0.386866316,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-214.ec2.internal,}"
Jul 01 14:39:10.457489 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.457469    2541 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvltc"
Jul 01 14:39:10.466130 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.466107    2541 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvltc"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470555    2541 manager.go:337] "Starting Device Plugin manager"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.470582    2541 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470591    2541 server.go:87] "Starting device plugin registration server"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470827    2541 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470840    2541 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470921    2541 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.470998    2541 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.471005    2541 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.471932    2541 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.471972    2541 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.478549    2541 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.479830    2541 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.479859    2541 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.479877    2541 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.479905    2541 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 14:39:10.484116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.482452    2541 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:10.571346 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.571308    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.572354 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.572337    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.572455 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.572367    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.572455 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.572378    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.572455 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.572416    2541 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.580680 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.580629    2541 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"]
Jul 01 14:39:10.580738 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.580694    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.582193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.582180    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.582257 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.582210    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.582257 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.582224    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.583761 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.583748    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.583866 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.583852    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.583914 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.583880    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.584487 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584468    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.584591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584500    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.584591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584515    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.584591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584472    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.584591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584570    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.584591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.584580    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.585427 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.585412    2541 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.585500 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.585431    2541 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-214.ec2.internal\": node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.585765 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.585748    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.585837 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.585779    2541 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:10.586547 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.586531    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:10.586663 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.586561    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:10.586663 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.586577    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:10.612048 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.612019    2541 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-214.ec2.internal\" not found" node="ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.612318 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.612302    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.612427 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.612323    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.612427 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.612341    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b33c2a96ad53e3015792631c1ed9d0e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-214.ec2.internal\" (UID: \"4b33c2a96ad53e3015792631c1ed9d0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.616405 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.616374    2541 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-214.ec2.internal\" not found" node="ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.667315 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.667292    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.712636 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712610    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.712705 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712661    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.712705 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712660    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.712768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712691    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b33c2a96ad53e3015792631c1ed9d0e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-214.ec2.internal\" (UID: \"4b33c2a96ad53e3015792631c1ed9d0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.712768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712731    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209734d3ac72a04c003de7947e5d1017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal\" (UID: \"209734d3ac72a04c003de7947e5d1017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.712768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.712759    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b33c2a96ad53e3015792631c1ed9d0e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-214.ec2.internal\" (UID: \"4b33c2a96ad53e3015792631c1ed9d0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.767931 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.767902    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.868619 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.868559    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:10.914823 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.914783    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.918747 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:10.918537    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:10.969084 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:10.969053    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:11.069534 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:11.069504    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:11.170092 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:11.170028    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:12.508977 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:12.508951    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:12.590479 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.590447    2541 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 14:39:12.590977 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.590604    2541 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 14:39:12.609742 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:12.609712    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:12.708320 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.708213    2541 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 14:34:11 +0000 UTC" deadline="2028-03-24 22:05:42.610334624 +0000 UTC"
Jul 01 14:39:12.708320 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.708265    2541 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15175h26m29.902074422s"
Jul 01 14:39:12.710379 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:12.710357    2541 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-214.ec2.internal\" not found"
Jul 01 14:39:12.718212 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.718194    2541 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:12.740280 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.739226    2541 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:12.750626 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.750595    2541 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:12.777162 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.777139    2541 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 14:39:12.778177 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.778162    2541 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal"
Jul 01 14:39:12.792946 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.792921    2541 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 14:39:12.887177 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:12.887134    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209734d3ac72a04c003de7947e5d1017.slice/crio-5afff04d38705b6422383ced6030b42881e9fbdd43c1b034b1486c5b483e3181 WatchSource:0}: Error finding container 5afff04d38705b6422383ced6030b42881e9fbdd43c1b034b1486c5b483e3181: Status 404 returned error can't find the container with id 5afff04d38705b6422383ced6030b42881e9fbdd43c1b034b1486c5b483e3181
Jul 01 14:39:12.887462 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:12.887438    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b33c2a96ad53e3015792631c1ed9d0e.slice/crio-3f13e1b23c5ea87f9ba55f03a47d20865bba635e9ecfd6f0878a8f4b5668702f WatchSource:0}: Error finding container 3f13e1b23c5ea87f9ba55f03a47d20865bba635e9ecfd6f0878a8f4b5668702f: Status 404 returned error can't find the container with id 3f13e1b23c5ea87f9ba55f03a47d20865bba635e9ecfd6f0878a8f4b5668702f
Jul 01 14:39:12.894034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.894017    2541 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 14:39:12.931405 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:12.931382    2541 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:13.477897 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.477866    2541 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:13.631345 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.631321    2541 apiserver.go:52] "Watching apiserver"
Jul 01 14:39:13.639018 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.638996    2541 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 14:39:13.640409 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.640375    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-fqlzc","openshift-multus/multus-k998v","kube-system/global-pull-secret-syncer-mwx8j","openshift-dns/node-resolver-rb9zw","openshift-multus/network-metrics-daemon-p5s2b","openshift-network-diagnostics/network-check-target-dz29r","openshift-network-operator/iptables-alerter-64w57","openshift-ovn-kubernetes/ovnkube-node-zh22p","kube-system/konnectivity-agent-l5t2l","kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k","openshift-cluster-node-tuning-operator/tuned-bfdpj","openshift-image-registry/node-ca-2jjvg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal"]
Jul 01 14:39:13.642749 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.642732    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.642852 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.642814    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:13.643727 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.643710    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.644775 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.644756    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.646184 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.646169    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.646284 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.646267    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m9tqs\""
Jul 01 14:39:13.646345 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.646331    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 14:39:13.647274 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647249    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.647387 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647288    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.647448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647401    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.647448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647417    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.647539 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647520    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 14:39:13.647600 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.647585    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rmjxk\""
Jul 01 14:39:13.648055 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.648037    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.648685 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.648651    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:13.648774 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.648736    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:13.649169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649033    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 14:39:13.649169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649053    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.649169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649059    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 14:39:13.649169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649134    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-phlgx\""
Jul 01 14:39:13.649512 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649496    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.649776 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649760    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 14:39:13.649888 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649876    2541 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 14:39:13.649952 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.649917    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.651341 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.651209    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.652624 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.652606    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.654032 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.654013    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.654271 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.654238    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:13.658678 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658659    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-systemd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.658753 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658694    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-registration-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.658753 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658719    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-device-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.658753 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658741    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t79d8\" (UniqueName: \"kubernetes.io/projected/90785c28-636d-4a29-b552-5f3ee29abbd8-kube-api-access-t79d8\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658757    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-system-cni-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658770    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-systemd-units\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658786    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658830    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-netd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658869    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658891    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.658918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658918    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2qch\" (UniqueName: \"kubernetes.io/projected/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-kube-api-access-t2qch\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658937    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658950    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-netns\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658963    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-ovn\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.658985    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-env-overrides\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659007    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659021    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftprb\" (UniqueName: \"kubernetes.io/projected/bb43336f-c51e-4d6f-8708-89f7e9713860-kube-api-access-ftprb\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659035    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-host-slash\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659048    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-slash\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659085    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-var-lib-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659121    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-log-socket\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659146    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-bin\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659169    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-host\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.659193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659191    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-serviceca\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659211    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659238    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659236    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjhx6\" (UniqueName: \"kubernetes.io/projected/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-kube-api-access-sjhx6\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659238    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2j24w\""
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659288    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-etc-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659307    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-config\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659321    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-script-lib\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659345    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659384    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-cnibin\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659407    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-os-release\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659426    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-iptables-alerter-script\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659443    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-hosts-file\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659457    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-tmp-dir\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659477    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659498    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659507    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659520    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.659763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659529    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659544    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c42jr\" (UniqueName: \"kubernetes.io/projected/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-kube-api-access-c42jr\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659565    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-kubelet\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659586    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-node-log\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659608    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovn-node-metrics-cert\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659631    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dglkx\" (UniqueName: \"kubernetes.io/projected/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-kube-api-access-dglkx\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659653    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-socket-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659697    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2kkl\" (UniqueName: \"kubernetes.io/projected/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-kube-api-access-p2kkl\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659699    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659736    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-sys-fs\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659750    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659760    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-binary-copy\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659833    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.659914    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660121    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660132    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.660388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660141    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.660922 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660745    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bjqkd\""
Jul 01 14:39:13.660922 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660755    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 14:39:13.660922 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660768    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 14:39:13.660922 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660797    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 14:39:13.660922 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660885    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c7rpj\""
Jul 01 14:39:13.661070 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.660997    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.661146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.661130    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9b5t4\""
Jul 01 14:39:13.661194 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.661133    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 14:39:13.664174 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.664153    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:13.664367 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.664353    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:13.665483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.665463    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 14:39:13.665592 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.665537    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fr5m7\""
Jul 01 14:39:13.665677 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.665653    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j88gg\""
Jul 01 14:39:13.672206 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.672188    2541 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 14:39:13.713898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.713873    2541 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dbvsx"
Jul 01 14:39:13.721419 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.721394    2541 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dbvsx"
Jul 01 14:39:13.722511 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.722441    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal" event={"ID":"4b33c2a96ad53e3015792631c1ed9d0e","Type":"ContainerStarted","Data":"3f13e1b23c5ea87f9ba55f03a47d20865bba635e9ecfd6f0878a8f4b5668702f"}
Jul 01 14:39:13.723605 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.723583    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal" event={"ID":"209734d3ac72a04c003de7947e5d1017","Type":"ContainerStarted","Data":"5afff04d38705b6422383ced6030b42881e9fbdd43c1b034b1486c5b483e3181"}
Jul 01 14:39:13.752770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.751854    2541 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 14:39:13.760349 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760322    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760351    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftprb\" (UniqueName: \"kubernetes.io/projected/bb43336f-c51e-4d6f-8708-89f7e9713860-kube-api-access-ftprb\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760378    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ck5d\" (UniqueName: \"kubernetes.io/projected/e9079c38-8718-4862-a6b6-c266032405c0-kube-api-access-7ck5d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760394    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-bin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760412    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-slash\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760429    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-var-lib-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760444    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-serviceca\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760458    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-conf\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760479    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-conf-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.760501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760476    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-slash\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760513    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-var-lib-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760542    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760564    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1de49db7-4331-4304-b09e-300573ca8724-konnectivity-ca\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760602    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-etc-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760620    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-config\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760637    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-script-lib\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760662    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-cnibin\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760670    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-etc-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760708    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-os-release\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760731    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-dbus\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760750    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-cnibin\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760790    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760802    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-os-release\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760815    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-tmp\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760882    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-hosts-file\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760909    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-tmp-dir\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.760980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760928    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760954    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760976    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-hosts-file\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760979    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760993    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-serviceca\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.760983    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-etc-tuned\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761060    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cnibin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761089    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-multus\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761114    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-kubelet\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761142    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-kubelet\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761165    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-node-log\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761191    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-socket-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761199    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-config\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761218    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2kkl\" (UniqueName: \"kubernetes.io/projected/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-kube-api-access-p2kkl\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761237    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovnkube-script-lib\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761243    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-node-log\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761245    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-tmp-dir\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.761718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761275    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-sys-fs\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761287    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-kubelet\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761296    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-binary-copy\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761303    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-socket-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761359    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-sys-fs\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761379    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-system-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761400    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-device-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761418    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-system-cni-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761450    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-kubelet-config\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761469    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-daemon-config\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761485    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761483    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-device-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761482    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761483    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb43336f-c51e-4d6f-8708-89f7e9713860-system-cni-dir\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761515    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysconfig\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761617    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-lib-modules\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761637    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-netns\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.762404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761675    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-ovn\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761700    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-host-slash\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761702    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-run-netns\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761675    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-binary-copy\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761743    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-ovn\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761743    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-host-slash\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761756    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-host\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761787    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cni-binary-copy\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761813    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-log-socket\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761855    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-bin\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761874    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-log-socket\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761880    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-host\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761901    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-bin\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761903    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761927    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-host\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761927    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjhx6\" (UniqueName: \"kubernetes.io/projected/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-kube-api-access-sjhx6\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761952    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-netns\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761967    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-hostroot\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.761986    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28p95\" (UniqueName: \"kubernetes.io/projected/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-kube-api-access-28p95\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762012    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762029    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-iptables-alerter-script\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762043    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-kubernetes\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762053    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762073    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-sys\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762094    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762107    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762130    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-var-lib-kubelet\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762159    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762181    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-k8s-cni-cncf-io\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762203    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-etc-kubernetes\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762227    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c42jr\" (UniqueName: \"kubernetes.io/projected/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-kube-api-access-c42jr\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762251    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovn-node-metrics-cert\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762273    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dglkx\" (UniqueName: \"kubernetes.io/projected/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-kube-api-access-dglkx\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762297    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762320    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-run\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.763594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762343    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-os-release\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762367    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-multus-certs\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762389    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-socket-dir-parent\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762414    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1de49db7-4331-4304-b09e-300573ca8724-agent-certs\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762448    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-systemd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762489    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-registration-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762506    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb43336f-c51e-4d6f-8708-89f7e9713860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762520    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-systemd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762521    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t79d8\" (UniqueName: \"kubernetes.io/projected/90785c28-636d-4a29-b552-5f3ee29abbd8-kube-api-access-t79d8\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762561    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-systemd-units\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762560    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-iptables-alerter-script\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762576    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762578    2541 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762593    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-netd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762561    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90785c28-636d-4a29-b552-5f3ee29abbd8-registration-dir\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762626    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-systemd-units\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762627    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762648    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-cni-netd\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762655    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762629    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-run-openvswitch\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762665    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762686    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2qch\" (UniqueName: \"kubernetes.io/projected/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-kube-api-access-t2qch\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762707    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-modprobe-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.762723    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762726    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-systemd\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.762752    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-env-overrides\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.762779    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:14.262761639 +0000 UTC m=+2.979146374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:13.764620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.763072    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-env-overrides\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.765652 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.765629    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-ovn-node-metrics-cert\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.771585 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.771563    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftprb\" (UniqueName: \"kubernetes.io/projected/bb43336f-c51e-4d6f-8708-89f7e9713860-kube-api-access-ftprb\") pod \"multus-additional-cni-plugins-fqlzc\" (UID: \"bb43336f-c51e-4d6f-8708-89f7e9713860\") " pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.773014 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.772994    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:13.773103 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.773017    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:13.773103 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.773057    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:13.773220 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.773127    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:14.273093683 +0000 UTC m=+2.989478443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:13.774559 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.774534    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t79d8\" (UniqueName: \"kubernetes.io/projected/90785c28-636d-4a29-b552-5f3ee29abbd8-kube-api-access-t79d8\") pod \"aws-ebs-csi-driver-node-8899k\" (UID: \"90785c28-636d-4a29-b552-5f3ee29abbd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.775104 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.775085    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjhx6\" (UniqueName: \"kubernetes.io/projected/d841bd2a-2624-4f19-9421-b2e0eed2a1d2-kube-api-access-sjhx6\") pod \"iptables-alerter-64w57\" (UID: \"d841bd2a-2624-4f19-9421-b2e0eed2a1d2\") " pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.775180 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.775139    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c42jr\" (UniqueName: \"kubernetes.io/projected/1cb87f50-27ec-4332-bb8d-5f33ffa8fa45-kube-api-access-c42jr\") pod \"node-resolver-rb9zw\" (UID: \"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45\") " pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.775464 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.775445    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2kkl\" (UniqueName: \"kubernetes.io/projected/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-kube-api-access-p2kkl\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:13.775524 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.775470    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2qch\" (UniqueName: \"kubernetes.io/projected/31c63ed0-9e3f-4e22-8844-f28c0d5b9425-kube-api-access-t2qch\") pod \"node-ca-2jjvg\" (UID: \"31c63ed0-9e3f-4e22-8844-f28c0d5b9425\") " pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.775726 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.775710    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dglkx\" (UniqueName: \"kubernetes.io/projected/8b2ff6ec-f4d3-4760-8695-385d1cbfab3d-kube-api-access-dglkx\") pod \"ovnkube-node-zh22p\" (UID: \"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.863817 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863776    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-modprobe-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.863817 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863811    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-systemd\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863865    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ck5d\" (UniqueName: \"kubernetes.io/projected/e9079c38-8718-4862-a6b6-c266032405c0-kube-api-access-7ck5d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863888    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-bin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863898    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-modprobe-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863910    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-conf\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863942    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-systemd\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863946    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-conf-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863974    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-bin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863987    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1de49db7-4331-4304-b09e-300573ca8724-konnectivity-ca\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.863976    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-conf-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864015    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-dbus\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864029    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864036    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-conf\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864067 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864051    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-tmp\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864089    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-etc-tuned\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864115    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cnibin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864138    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-multus\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864157    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysctl-d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864162    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-kubelet\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864199    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-system-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864225    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-kubelet-config\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864235    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-kubelet\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864241    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-system-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864246    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cnibin\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864200    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-var-lib-cni-multus\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864273    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-daemon-config\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864297    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-kubelet-config\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864311    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysconfig\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864318    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-dbus\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864335    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-lib-modules\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864361    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-host\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.864716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864377    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-sysconfig\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864388    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cni-binary-copy\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864418    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-netns\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864445    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-host\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864446    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-hostroot\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864485    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-hostroot\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864511    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-lib-modules\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864532    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-netns\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864486    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28p95\" (UniqueName: \"kubernetes.io/projected/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-kube-api-access-28p95\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864570    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-kubernetes\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864581    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1de49db7-4331-4304-b09e-300573ca8724-konnectivity-ca\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864592    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-sys\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864628    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-var-lib-kubelet\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864658    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864683    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-k8s-cni-cncf-io\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864687    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-sys\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864706    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-etc-kubernetes\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864730    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-etc-kubernetes\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.865522 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864741    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864767    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-k8s-cni-cncf-io\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864787    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-daemon-config\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864803    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-etc-kubernetes\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864810    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-run\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864768    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-run\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864866    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-os-release\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864884    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9079c38-8718-4862-a6b6-c266032405c0-var-lib-kubelet\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.864889    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864897    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-cni-dir\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864921    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-multus-certs\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864891    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-host-run-multus-certs\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864947    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-os-release\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:13.864952    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:14.364935911 +0000 UTC m=+3.081320671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.864993    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-socket-dir-parent\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.865019    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1de49db7-4331-4304-b09e-300573ca8724-agent-certs\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.865059    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-multus-socket-dir-parent\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.865181    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-cni-binary-copy\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.866933 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.866425    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-etc-tuned\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.866933 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.866469    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9079c38-8718-4862-a6b6-c266032405c0-tmp\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.867247 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.867229    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1de49db7-4331-4304-b09e-300573ca8724-agent-certs\") pod \"konnectivity-agent-l5t2l\" (UID: \"1de49db7-4331-4304-b09e-300573ca8724\") " pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.872521 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.872496    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ck5d\" (UniqueName: \"kubernetes.io/projected/e9079c38-8718-4862-a6b6-c266032405c0-kube-api-access-7ck5d\") pod \"tuned-bfdpj\" (UID: \"e9079c38-8718-4862-a6b6-c266032405c0\") " pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.873228 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.873208    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28p95\" (UniqueName: \"kubernetes.io/projected/5e28982b-cd07-4630-92f5-0f4ce06b6ce1-kube-api-access-28p95\") pod \"multus-k998v\" (UID: \"5e28982b-cd07-4630-92f5-0f4ce06b6ce1\") " pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.954008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.953976    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jjvg"
Jul 01 14:39:13.960931 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.960916    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k"
Jul 01 14:39:13.963376 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.963355    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c63ed0_9e3f_4e22_8844_f28c0d5b9425.slice/crio-93c9bc56e3de77eb21034f39c4ff0e8bcc623edbfa85296721d79d6f56be51be WatchSource:0}: Error finding container 93c9bc56e3de77eb21034f39c4ff0e8bcc623edbfa85296721d79d6f56be51be: Status 404 returned error can't find the container with id 93c9bc56e3de77eb21034f39c4ff0e8bcc623edbfa85296721d79d6f56be51be
Jul 01 14:39:13.968171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.968150    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fqlzc"
Jul 01 14:39:13.968428 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.968410    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90785c28_636d_4a29_b552_5f3ee29abbd8.slice/crio-1d5bc83f96934d5a25bab28608433329b8bd581cda99d35cd42e5196e0ac0b1d WatchSource:0}: Error finding container 1d5bc83f96934d5a25bab28608433329b8bd581cda99d35cd42e5196e0ac0b1d: Status 404 returned error can't find the container with id 1d5bc83f96934d5a25bab28608433329b8bd581cda99d35cd42e5196e0ac0b1d
Jul 01 14:39:13.973163 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.973147    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-64w57"
Jul 01 14:39:13.976586 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.976566    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb43336f_c51e_4d6f_8708_89f7e9713860.slice/crio-aa420e26b4841ce572dc5f246964e6cb8bedf3e4d2d8abdb759d29851f7a25e0 WatchSource:0}: Error finding container aa420e26b4841ce572dc5f246964e6cb8bedf3e4d2d8abdb759d29851f7a25e0: Status 404 returned error can't find the container with id aa420e26b4841ce572dc5f246964e6cb8bedf3e4d2d8abdb759d29851f7a25e0
Jul 01 14:39:13.977933 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.977872    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rb9zw"
Jul 01 14:39:13.981740 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.981719    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd841bd2a_2624_4f19_9421_b2e0eed2a1d2.slice/crio-518febb6e948642c394062b2d26f172ea0c135d6c43ea64e64f98455747d2718 WatchSource:0}: Error finding container 518febb6e948642c394062b2d26f172ea0c135d6c43ea64e64f98455747d2718: Status 404 returned error can't find the container with id 518febb6e948642c394062b2d26f172ea0c135d6c43ea64e64f98455747d2718
Jul 01 14:39:13.983075 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.983056    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:13.985650 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.985621    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb87f50_27ec_4332_bb8d_5f33ffa8fa45.slice/crio-27262ace3fda8980e1925d548f92c444ed7aa9bd1787bddc7d9b084c9616ef66 WatchSource:0}: Error finding container 27262ace3fda8980e1925d548f92c444ed7aa9bd1787bddc7d9b084c9616ef66: Status 404 returned error can't find the container with id 27262ace3fda8980e1925d548f92c444ed7aa9bd1787bddc7d9b084c9616ef66
Jul 01 14:39:13.989144 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.989125    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:13.990608 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.990591    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2ff6ec_f4d3_4760_8695_385d1cbfab3d.slice/crio-5e57696bcc15e7673d03f3d58cff8762f824285b6603259e9804a7dfa5e48cca WatchSource:0}: Error finding container 5e57696bcc15e7673d03f3d58cff8762f824285b6603259e9804a7dfa5e48cca: Status 404 returned error can't find the container with id 5e57696bcc15e7673d03f3d58cff8762f824285b6603259e9804a7dfa5e48cca
Jul 01 14:39:13.994731 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.994711    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bfdpj"
Jul 01 14:39:13.998408 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:13.998390    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k998v"
Jul 01 14:39:13.998658 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:13.998635    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de49db7_4331_4304_b09e_300573ca8724.slice/crio-dad2ae7a08a7419516260899edcbbebfdc14f1d0d4f3ae657c3b1d9eaffb59b4 WatchSource:0}: Error finding container dad2ae7a08a7419516260899edcbbebfdc14f1d0d4f3ae657c3b1d9eaffb59b4: Status 404 returned error can't find the container with id dad2ae7a08a7419516260899edcbbebfdc14f1d0d4f3ae657c3b1d9eaffb59b4
Jul 01 14:39:14.002851 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:14.002786    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9079c38_8718_4862_a6b6_c266032405c0.slice/crio-629b06a0efdd693584b97164a785c92f56706c8a0c636bc71436ba660fd25b64 WatchSource:0}: Error finding container 629b06a0efdd693584b97164a785c92f56706c8a0c636bc71436ba660fd25b64: Status 404 returned error can't find the container with id 629b06a0efdd693584b97164a785c92f56706c8a0c636bc71436ba660fd25b64
Jul 01 14:39:14.006855 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:14.006821    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e28982b_cd07_4630_92f5_0f4ce06b6ce1.slice/crio-5a5bb7e64f263ba3e62b6c7c16ad86e8d394151a52aefb96af183b47d4ad350c WatchSource:0}: Error finding container 5a5bb7e64f263ba3e62b6c7c16ad86e8d394151a52aefb96af183b47d4ad350c: Status 404 returned error can't find the container with id 5a5bb7e64f263ba3e62b6c7c16ad86e8d394151a52aefb96af183b47d4ad350c
Jul 01 14:39:14.267422 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.267330    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:14.267567 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.267498    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:14.267622 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.267569    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:15.267553151 +0000 UTC m=+3.983937886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:14.368243 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.368200    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:14.368417 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.368265    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:14.368417 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368406    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:14.368527 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368463    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:15.368445935 +0000 UTC m=+4.084830676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:14.368589 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368549    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:14.368589 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368563    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:14.368589 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368575    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:14.368741 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:14.368608    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:15.368597879 +0000 UTC m=+4.084982616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:14.724378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.724321    2541 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 14:34:13 +0000 UTC" deadline="2027-12-06 06:37:06.014442772 +0000 UTC"
Jul 01 14:39:14.724378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.724375    2541 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12543h57m51.290071637s"
Jul 01 14:39:14.740987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.740949    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k998v" event={"ID":"5e28982b-cd07-4630-92f5-0f4ce06b6ce1","Type":"ContainerStarted","Data":"5a5bb7e64f263ba3e62b6c7c16ad86e8d394151a52aefb96af183b47d4ad350c"}
Jul 01 14:39:14.749712 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.749419    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bfdpj" event={"ID":"e9079c38-8718-4862-a6b6-c266032405c0","Type":"ContainerStarted","Data":"629b06a0efdd693584b97164a785c92f56706c8a0c636bc71436ba660fd25b64"}
Jul 01 14:39:14.752739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.752640    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rb9zw" event={"ID":"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45","Type":"ContainerStarted","Data":"27262ace3fda8980e1925d548f92c444ed7aa9bd1787bddc7d9b084c9616ef66"}
Jul 01 14:39:14.755435 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.755359    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-64w57" event={"ID":"d841bd2a-2624-4f19-9421-b2e0eed2a1d2","Type":"ContainerStarted","Data":"518febb6e948642c394062b2d26f172ea0c135d6c43ea64e64f98455747d2718"}
Jul 01 14:39:14.760087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.760062    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k" event={"ID":"90785c28-636d-4a29-b552-5f3ee29abbd8","Type":"ContainerStarted","Data":"1d5bc83f96934d5a25bab28608433329b8bd581cda99d35cd42e5196e0ac0b1d"}
Jul 01 14:39:14.762949 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.762761    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5t2l" event={"ID":"1de49db7-4331-4304-b09e-300573ca8724","Type":"ContainerStarted","Data":"dad2ae7a08a7419516260899edcbbebfdc14f1d0d4f3ae657c3b1d9eaffb59b4"}
Jul 01 14:39:14.770279 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.770192    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"5e57696bcc15e7673d03f3d58cff8762f824285b6603259e9804a7dfa5e48cca"}
Jul 01 14:39:14.773798 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.773744    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerStarted","Data":"aa420e26b4841ce572dc5f246964e6cb8bedf3e4d2d8abdb759d29851f7a25e0"}
Jul 01 14:39:14.783678 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:14.783600    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jjvg" event={"ID":"31c63ed0-9e3f-4e22-8844-f28c0d5b9425","Type":"ContainerStarted","Data":"93c9bc56e3de77eb21034f39c4ff0e8bcc623edbfa85296721d79d6f56be51be"}
Jul 01 14:39:15.276973 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.276924    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:15.277162 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.277077    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:15.277162 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.277136    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:17.277118448 +0000 UTC m=+5.993503188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:15.377964 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.377927    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:15.378156 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.377992    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:15.378156 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378130    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:15.378269 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378196    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:17.378174059 +0000 UTC m=+6.094558812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:15.378627 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378609    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:15.378742 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378634    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:15.378742 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378647    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:15.378742 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.378701    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:17.378686888 +0000 UTC m=+6.095071641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:15.721651 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.721576    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:15.721805 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.721729    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:15.722158 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.722142    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:15.722282 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.722231    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:15.722344 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.722297    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:15.722401 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:15.722369    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:15.724591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.724528    2541 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 14:34:13 +0000 UTC" deadline="2028-02-16 12:38:41.138023359 +0000 UTC"
Jul 01 14:39:15.724591 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:15.724563    2541 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14277h59m25.413464553s"
Jul 01 14:39:17.295253 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.295214    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:17.295832 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.295362    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:17.295832 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.295429    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:21.295408707 +0000 UTC m=+10.011793458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:17.396389 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.396349    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:17.396555 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.396416    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:17.396636 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.396559    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:17.396636 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.396618    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:21.396600546 +0000 UTC m=+10.112985299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:17.397095 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.397074    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:17.397196 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.397102    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:17.397196 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.397115    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:17.397196 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.397158    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:21.397143857 +0000 UTC m=+10.113528595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:17.721036 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.720957    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:17.721198 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.721091    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:17.721520 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.721500    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:17.721629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.721614    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:17.721692 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:17.721645    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:17.721780 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:17.721747    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:19.721006 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:19.720734    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:19.721502 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:19.720952    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:19.721502 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:19.721105    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:19.721502 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:19.721143    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:19.721502 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:19.721266    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:19.721502 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:19.721351    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:21.332916 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.332875    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:21.333312 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.333061    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:21.333312 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.333143    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:29.333122294 +0000 UTC m=+18.049507040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.433540    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.433636    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.433771    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.433787    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.433796    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.433869    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:29.433852726 +0000 UTC m=+18.150237474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.434249    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:21.434323 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.434286    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:29.434275438 +0000 UTC m=+18.150660174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:21.722434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.722365    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:21.722580 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.722488    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:21.723394 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.722863    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:21.723394 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.722969    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:21.723394 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:21.723014    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:21.723394 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:21.723078    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:23.721557 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:23.721518    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:23.721971 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:23.721654    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:23.721971 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:23.721963    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:23.722083 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:23.722063    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:23.722141 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:23.722119    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:23.722263 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:23.722241    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:25.718666 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:25.718641    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:25.719116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:25.718670    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:25.719116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:25.718646    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:25.719116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:25.718754    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:25.719116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:25.718827    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:25.719116 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:25.718915    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:27.719541 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:27.719497    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:27.720024 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:27.719497    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:27.720024 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:27.719636    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:27.720024 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:27.719506    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:27.720024 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:27.719833    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:27.720024 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:27.719965    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:29.389384 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.389333    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:29.389959 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.389498    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:29.389959 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.389568    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:39:45.389551141 +0000 UTC m=+34.105935896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:29.490459 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.490421    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:29.490640 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.490517    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:29.490640 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490596    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:29.490754 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490657    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:29.490754 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490672    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:45.490652851 +0000 UTC m=+34.207037586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:29.490754 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490676    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:29.490754 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490691    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:29.490754 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.490743    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:39:45.490728918 +0000 UTC m=+34.207113659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:29.719449 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.719354    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:29.719449 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.719415    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:29.719666 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:29.719359    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:29.719666 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.719506    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:29.719666 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.719606    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:29.719836 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:29.719717    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:31.720376 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:31.720344    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:31.720831 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:31.720465    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:31.720831 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:31.720474    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:31.720831 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:31.720583    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:31.720831 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:31.720625    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:31.720831 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:31.720692    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:33.719673 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:33.719634    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:33.720198 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:33.719709    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:33.720198 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:33.719715    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:33.720198 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:33.719804    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:33.720198 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:33.719911    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:33.720198 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:33.720001    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:34.827001 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.826766    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827270    2541 generic.go:356] "Generic (PLEG): container finished" podID="8b2ff6ec-f4d3-4760-8695-385d1cbfab3d" containerID="20f100704919fea1bcac480bd5cdbfdaba4143d0ef5005eea88086e82a52cd51" exitCode=1
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827356    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"e9fe46ffe0dbc1f92b86e147a3dc630cfaebc1eabe88d4fd6f8df9398f51ab06"}
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827463    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"0e14e32f71ef5b5090c37222c3763148fb3a011f94dadf1fa2830a97de818492"}
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827507    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"ca1b08e465a244de5b63c4e05d06cd9a8be59561ac3402b49a6ef10347166f54"}
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827551    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"a6264b4a78935eb3b2b50871ab99b0b57e5ee1a3015e173f6e6b9bc95486302a"}
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827596    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerDied","Data":"20f100704919fea1bcac480bd5cdbfdaba4143d0ef5005eea88086e82a52cd51"}
Jul 01 14:39:34.827978 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.827637    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"a237fb488ca601b5285c9ce4f0c8e4c932879edb40ddb951e2126807d37104c8"}
Jul 01 14:39:34.829096 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.829072    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal" event={"ID":"4b33c2a96ad53e3015792631c1ed9d0e","Type":"ContainerStarted","Data":"29b3abef3daa3ff6b39035058ee03430fa367f739cd7bf665107d6acabf4c0d8"}
Jul 01 14:39:34.831082 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.831059    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k998v" event={"ID":"5e28982b-cd07-4630-92f5-0f4ce06b6ce1","Type":"ContainerStarted","Data":"0927292036bdf910667bac084a6453d162b9a4417bf6fa471ef5b93faecee0f1"}
Jul 01 14:39:34.832361 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.832342    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bfdpj" event={"ID":"e9079c38-8718-4862-a6b6-c266032405c0","Type":"ContainerStarted","Data":"e50257c3d420cef304286b96478647ca95378179ed80c7361a30e1302553a32d"}
Jul 01 14:39:34.877878 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.877798    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-214.ec2.internal" podStartSLOduration=22.87777913 podStartE2EDuration="22.87777913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:34.848919565 +0000 UTC m=+23.565304345" watchObservedRunningTime="2026-07-01 14:39:34.87777913 +0000 UTC m=+23.594163968"
Jul 01 14:39:34.908071 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.908022    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k998v" podStartSLOduration=2.935349203 podStartE2EDuration="22.908004914s" totalImagesPullingTime="19.972655711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:34.906887506 +0000 UTC m=+23.623272263" watchObservedRunningTime="2026-07-01 14:39:34.908004914 +0000 UTC m=+23.624389671"
Jul 01 14:39:34.908732 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:34.908701    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bfdpj" podStartSLOduration=2.990070145 podStartE2EDuration="22.908688643s" totalImagesPullingTime="19.918618498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:34.878197865 +0000 UTC m=+23.594582622" watchObservedRunningTime="2026-07-01 14:39:34.908688643 +0000 UTC m=+23.625073401"
Jul 01 14:39:35.719510 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.719320    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:35.719813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.719321    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:35.719813 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:35.719602    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:35.719813 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:35.719698    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:35.719813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.719321    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:35.719813 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:35.719809    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:35.836790 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.836754    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5t2l" event={"ID":"1de49db7-4331-4304-b09e-300573ca8724","Type":"ContainerStarted","Data":"36be3915ea2f6914f97235a29df3039bac257bd9c163f1076b97f746d637b32f"}
Jul 01 14:39:35.838597 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.838573    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="d5f7fed47d54f0cf7a8f752505b47a68d494487f275efd5939145a5026583f24" exitCode=0
Jul 01 14:39:35.838722 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.838659    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"d5f7fed47d54f0cf7a8f752505b47a68d494487f275efd5939145a5026583f24"}
Jul 01 14:39:35.840190 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.840157    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jjvg" event={"ID":"31c63ed0-9e3f-4e22-8844-f28c0d5b9425","Type":"ContainerStarted","Data":"46b0cc53f87900aace1560e6a5f59ee4e86f728d4d8323f3894a8acd02a1efb0"}
Jul 01 14:39:35.841936 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.841916    2541 generic.go:356] "Generic (PLEG): container finished" podID="209734d3ac72a04c003de7947e5d1017" containerID="eb21fe9c150dfe15b1d97fda13d727f5fe525fdafd17fbf1ee7b90b612e8defe" exitCode=0
Jul 01 14:39:35.842024 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.842008    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal" event={"ID":"209734d3ac72a04c003de7947e5d1017","Type":"ContainerDied","Data":"eb21fe9c150dfe15b1d97fda13d727f5fe525fdafd17fbf1ee7b90b612e8defe"}
Jul 01 14:39:35.843570 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.843546    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rb9zw" event={"ID":"1cb87f50-27ec-4332-bb8d-5f33ffa8fa45","Type":"ContainerStarted","Data":"d0c92cd9fc6a25b7d1281ab2715c0880b3294596115d5822444a5bd7043d0c3d"}
Jul 01 14:39:35.845204 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.845184    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-64w57" event={"ID":"d841bd2a-2624-4f19-9421-b2e0eed2a1d2","Type":"ContainerStarted","Data":"0f2f2b4085540b615fa470a1d672d9549e2a417f055f1c5bb164df229506b335"}
Jul 01 14:39:35.846619 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.846598    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k" event={"ID":"90785c28-636d-4a29-b552-5f3ee29abbd8","Type":"ContainerStarted","Data":"d50c7b99718e87efcf8af1cc05043d00098428ebfd19cf18ddf2093662f03679"}
Jul 01 14:39:35.858941 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.858896    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l5t2l" podStartSLOduration=3.937457233 podStartE2EDuration="23.858882886s" totalImagesPullingTime="19.921425653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:35.858163076 +0000 UTC m=+24.574547834" watchObservedRunningTime="2026-07-01 14:39:35.858882886 +0000 UTC m=+24.575267642"
Jul 01 14:39:35.894436 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.894393    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rb9zw" podStartSLOduration=3.9594856959999998 podStartE2EDuration="23.89437575s" totalImagesPullingTime="19.934890054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:35.893041492 +0000 UTC m=+24.609426251" watchObservedRunningTime="2026-07-01 14:39:35.89437575 +0000 UTC m=+24.610760510"
Jul 01 14:39:35.930272 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:35.930029    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2jjvg" podStartSLOduration=9.32974418 podStartE2EDuration="24.930011351s" totalImagesPullingTime="15.600267171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:35.929887767 +0000 UTC m=+24.646272527" watchObservedRunningTime="2026-07-01 14:39:35.930011351 +0000 UTC m=+24.646396109"
Jul 01 14:39:36.257593 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.257568    2541 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 14:39:36.721664 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.721555    2541 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T14:39:36.25758745Z","UUID":"3c80c5d7-08b5-48ba-aff1-f793bae40cd3","Handler":null,"Name":"","Endpoint":""}
Jul 01 14:39:36.723327 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.723299    2541 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 14:39:36.723468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.723342    2541 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 14:39:36.850917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.850881    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal" event={"ID":"209734d3ac72a04c003de7947e5d1017","Type":"ContainerStarted","Data":"086c43fd729d7c8046ad16d87ed85366f2ce6e6538fc19a0d3907edcbb465316"}
Jul 01 14:39:36.853214 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.853187    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k" event={"ID":"90785c28-636d-4a29-b552-5f3ee29abbd8","Type":"ContainerStarted","Data":"f71c7f4fcaf2c128f41d3b1f3da51f68dab22bf8f2dbc50d54d6bcabcdfc4c13"}
Jul 01 14:39:36.874778 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.874737    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-64w57" podStartSLOduration=9.293059395 podStartE2EDuration="24.874716544s" totalImagesPullingTime="15.581657149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:36.021011712 +0000 UTC m=+24.737396470" watchObservedRunningTime="2026-07-01 14:39:36.874716544 +0000 UTC m=+25.591101304"
Jul 01 14:39:36.875224 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:36.875195    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-214.ec2.internal" podStartSLOduration=24.875183928 podStartE2EDuration="24.875183928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:36.874857534 +0000 UTC m=+25.591242286" watchObservedRunningTime="2026-07-01 14:39:36.875183928 +0000 UTC m=+25.591568686"
Jul 01 14:39:37.527090 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.527057    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:37.527773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.527753    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:37.719479 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.719449    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:37.719630 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:37.719591    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:37.720023 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.720005    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:37.720114 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:37.720092    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:37.720271 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.720248    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:37.720383 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:37.720363    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:37.858385 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.858293    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k" event={"ID":"90785c28-636d-4a29-b552-5f3ee29abbd8","Type":"ContainerStarted","Data":"0a9032393a1793b6d6bf7be3e5bf370151253536d27c4ffebca4f966e84a2cae"}
Jul 01 14:39:37.862023 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.861990    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:39:37.862460 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.862435    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"f093cfe53794d1a4dcce715ed931181d320e25be3c575ea7a427ea5b25ef74f2"}
Jul 01 14:39:37.879821 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:37.879766    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8899k" podStartSLOduration=4.43111919 podStartE2EDuration="26.879739489s" totalImagesPullingTime="22.448620299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:37.878733058 +0000 UTC m=+26.595117812" watchObservedRunningTime="2026-07-01 14:39:37.879739489 +0000 UTC m=+26.596124246"
Jul 01 14:39:38.436393 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:38.436358    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:38.437103 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:38.436869    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l5t2l"
Jul 01 14:39:39.718720 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:39.718687    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:39.719214 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:39.718687    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:39.719214 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:39.718817    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:39.719214 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:39.718930    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:39.719214 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:39.718687    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:39.719214 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:39.719032    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:40.871524 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.871356    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:39:40.872383 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.871870    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"eabde887b424893555850a46172e5b0611fa1286d3957b6e52a8b0d0f15f5e77"}
Jul 01 14:39:40.872383 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.872119    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:40.872383 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.872358    2541 scope.go:117] "RemoveContainer" containerID="20f100704919fea1bcac480bd5cdbfdaba4143d0ef5005eea88086e82a52cd51"
Jul 01 14:39:40.874057 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.873986    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="8e519341cd59a64abeb155526bc16f706030b700a2a2926718993ec4564f1868" exitCode=0
Jul 01 14:39:40.874057 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.874020    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"8e519341cd59a64abeb155526bc16f706030b700a2a2926718993ec4564f1868"}
Jul 01 14:39:40.889863 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:40.889826    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:41.719298 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.719268    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:41.719528 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:41.719350    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:41.719528 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.719454    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:41.719625 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:41.719581    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:41.719625 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.719612    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:41.719695 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:41.719682    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:41.880664 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.880630    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:39:41.881134 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.881041    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" event={"ID":"8b2ff6ec-f4d3-4760-8695-385d1cbfab3d","Type":"ContainerStarted","Data":"16a2bd28f0f1d4321851efcca8d96c463e78b7110bd6a887ab1ba10fdcba5d96"}
Jul 01 14:39:41.881421 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.881400    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:41.881580 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.881479    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:41.898307 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.898274    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:39:41.913672 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:41.913395    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p" podStartSLOduration=9.921346981 podStartE2EDuration="29.913370314s" totalImagesPullingTime="19.992023333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:41.911418561 +0000 UTC m=+30.627803320" watchObservedRunningTime="2026-07-01 14:39:41.913370314 +0000 UTC m=+30.629755120"
Jul 01 14:39:42.254223 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.254150    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dz29r"]
Jul 01 14:39:42.254407 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.254303    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:42.254461 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:42.254395    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:42.257701 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.257672    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p5s2b"]
Jul 01 14:39:42.257823 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.257791    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:42.257923 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:42.257901    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:42.258304 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.258277    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mwx8j"]
Jul 01 14:39:42.258417 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.258374    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:42.258493 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:42.258469    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:42.885972 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.885936    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="1e8080585998d447fef655cda5a1bcbed3fe5db8d613ccef1abb3feb13a8e283" exitCode=0
Jul 01 14:39:42.886437 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:42.885983    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"1e8080585998d447fef655cda5a1bcbed3fe5db8d613ccef1abb3feb13a8e283"}
Jul 01 14:39:43.718653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:43.718616    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:43.718653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:43.718649    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:43.718933 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:43.718616    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:43.718933 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:43.718733    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:43.718933 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:43.718796    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:43.718933 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:43.718901    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:44.892813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:44.892770    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="95343b38aaff079c89979b9023c2d4c5112bd1b99ab3fc99ffe0efcd05ffb944" exitCode=0
Jul 01 14:39:44.892813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:44.892815    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"95343b38aaff079c89979b9023c2d4c5112bd1b99ab3fc99ffe0efcd05ffb944"}
Jul 01 14:39:45.407341 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.407297    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:45.407521 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.407417    2541 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:45.407521 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.407470    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs podName:3500fb9f-71cb-4c7d-a1f7-9d58da2185eb nodeName:}" failed. No retries permitted until 2026-07-01 14:40:17.407456274 +0000 UTC m=+66.123841012 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs") pod "network-metrics-daemon-p5s2b" (UID: "3500fb9f-71cb-4c7d-a1f7-9d58da2185eb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:45.508507 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.508463    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:45.508691 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.508549    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:45.508691 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508627    2541 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:45.508691 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508673    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:45.508691 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508689    2541 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:45.508873 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508701    2541 projected.go:196] Error preparing data for projected volume kube-api-access-b4lcq for pod openshift-network-diagnostics/network-check-target-dz29r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:45.508873 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508707    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret podName:9143e78e-04cd-4a6b-8fd3-a25bf0aa0836 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:17.508683953 +0000 UTC m=+66.225068691 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret") pod "global-pull-secret-syncer-mwx8j" (UID: "9143e78e-04cd-4a6b-8fd3-a25bf0aa0836") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:45.508873 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.508751    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq podName:0c67366f-5b37-478a-911f-0d0fe521267b nodeName:}" failed. No retries permitted until 2026-07-01 14:40:17.508734881 +0000 UTC m=+66.225119633 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4lcq" (UniqueName: "kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq") pod "network-check-target-dz29r" (UID: "0c67366f-5b37-478a-911f-0d0fe521267b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:45.719118 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.719084    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:45.719277 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.719087    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:45.719277 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.719206    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dz29r" podUID="0c67366f-5b37-478a-911f-0d0fe521267b"
Jul 01 14:39:45.719408 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:45.719380    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:45.719534 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.719488    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5s2b" podUID="3500fb9f-71cb-4c7d-a1f7-9d58da2185eb"
Jul 01 14:39:45.719631 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:45.719603    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mwx8j" podUID="9143e78e-04cd-4a6b-8fd3-a25bf0aa0836"
Jul 01 14:39:47.070790 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.070682    2541 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-214.ec2.internal" event="NodeReady"
Jul 01 14:39:47.071231 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.070866    2541 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 14:39:47.144857 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.144818    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:39:47.146826 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.146797    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.174694 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.174656    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 14:39:47.176008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.175113    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"]
Jul 01 14:39:47.176987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.176967    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 14:39:47.177314 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.177291    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 14:39:47.177530 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.177506    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.178623 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.178595    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45ph4\""
Jul 01 14:39:47.180483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.180453    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-7qj2r"]
Jul 01 14:39:47.182780 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.182757    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.184890 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.184866    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.185329 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.185254    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.185329 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.185269    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 01 14:39:47.185329 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.185335    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4n8bh\""
Jul 01 14:39:47.185973 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.185953    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"]
Jul 01 14:39:47.188523 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.188290    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"]
Jul 01 14:39:47.188769 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.188736    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.190399 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.190383    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"]
Jul 01 14:39:47.190519 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.190502    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.192223 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.192205    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b66db9b69-6qgkk"]
Jul 01 14:39:47.192505 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.192487    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.194056 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.194034    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"]
Jul 01 14:39:47.194211 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.194195    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.195797 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.195768    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"
Jul 01 14:39:47.197292 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.197009    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 01 14:39:47.197292 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.197164    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.197292 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.197175    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 01 14:39:47.202809 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.202788    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 14:39:47.205069 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.205051    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 01 14:39:47.205255 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.205099    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 01 14:39:47.206055 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.206034    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 01 14:39:47.207063 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.207045    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jl6wx\""
Jul 01 14:39:47.207658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.207644    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.208284 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.208262    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.208385 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.208285    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.208459 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.208269    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.222422 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222399    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222536 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222437    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222536 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222470    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222563    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222600    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222666    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222687    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzg67\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.222768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.222720    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.223493 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.223471    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.224265    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cdd9t\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.224670    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.224888    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.225217    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.225441    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.225902    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.225984    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.226034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.226004    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.226590    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fbbrf\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.226671    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.226905    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.226983    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xslr9\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.227003    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"]
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.227023    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.227067    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 01 14:39:47.227171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.227154    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6shwn\""
Jul 01 14:39:47.228000 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.227984    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:39:47.228864 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.228830    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 01 14:39:47.235391 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.235367    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-7qj2r"]
Jul 01 14:39:47.236781 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.236762    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 01 14:39:47.245358 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.245330    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"]
Jul 01 14:39:47.245491 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.245460    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cqz7p\""
Jul 01 14:39:47.246319 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.246297    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"]
Jul 01 14:39:47.246684 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.246653    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 01 14:39:47.275328 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.275292    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"]
Jul 01 14:39:47.277221 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.277196    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b66db9b69-6qgkk"]
Jul 01 14:39:47.278109 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.278085    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"]
Jul 01 14:39:47.309655 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.309619    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nlnd6"]
Jul 01 14:39:47.312223 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.312200    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.323390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323324    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dh55\" (UniqueName: \"kubernetes.io/projected/70317fe5-e0e4-43f4-a19a-483c4b33fd40-kube-api-access-4dh55\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.323390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323353    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46mhr\" (UniqueName: \"kubernetes.io/projected/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-kube-api-access-46mhr\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.323390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323383    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.323573 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323419    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-stats-auth\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.323573 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323458    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-tmp\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.323573 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323479    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a922e63-f7a9-4190-9acc-20b9c68974cd-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.323573 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.323500    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:47.323573 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.323518    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323582    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.323602    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:47.823579433 +0000 UTC m=+36.539964185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323632    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mq4d\" (UniqueName: \"kubernetes.io/projected/4a922e63-f7a9-4190-9acc-20b9c68974cd-kube-api-access-8mq4d\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323652    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn5th\" (UniqueName: \"kubernetes.io/projected/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-kube-api-access-pn5th\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323684    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-serving-cert\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.323739 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323721    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a922e63-f7a9-4190-9acc-20b9c68974cd-config\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323744    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323785    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323806    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-default-certificate\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323830    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323880    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdx9s\" (UniqueName: \"kubernetes.io/projected/d309b969-9fb4-412d-b9c9-b793a68700c0-kube-api-access-tdx9s\") pod \"volume-data-source-validator-69d9f679db-f4vss\" (UID: \"d309b969-9fb4-412d-b9c9-b793a68700c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323923    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323955    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzg67\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.323981    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324000    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-snapshots\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.324016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324015    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-service-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324030    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324056    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324080    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324141    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324172    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324209    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrp27\" (UniqueName: \"kubernetes.io/projected/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-kube-api-access-nrp27\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324242    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324268    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s769\" (UniqueName: \"kubernetes.io/projected/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-kube-api-access-5s769\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324293    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324472 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324321    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324874 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324507    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324874 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324672    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.324874 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.324825    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.328811 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.328707    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.328950 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.328777    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.336154 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.336133    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 14:39:47.352564 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.352518    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs5sl\""
Jul 01 14:39:47.353382 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.353355    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"]
Jul 01 14:39:47.354786 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.354762    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 14:39:47.355836 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.355812    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-zgshx"]
Jul 01 14:39:47.355953 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.355941    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.357690 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.357671    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qhzkw"]
Jul 01 14:39:47.357799 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.357783    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx"
Jul 01 14:39:47.359683 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.359663    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"]
Jul 01 14:39:47.359777 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.359765    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.361589 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.361571    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.364515 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.364492    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kn8p7\""
Jul 01 14:39:47.364515 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.364509    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 01 14:39:47.365114 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.365093    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 01 14:39:47.365202 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.365173    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4rxw7\""
Jul 01 14:39:47.365393 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.365291    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.365460 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.365412    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 14:39:47.365781 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.365762    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.366035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366019    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.366086 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366069    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 01 14:39:47.366295 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366277    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.366404 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366389    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 14:39:47.366491 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366390    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 14:39:47.366491 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366430    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 01 14:39:47.366491 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366493    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-58rmg\""
Jul 01 14:39:47.366750 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.366614    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j49j4\""
Jul 01 14:39:47.383374 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.382304    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.386736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.386708    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlnd6"]
Jul 01 14:39:47.386972 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.386958    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qhzkw"]
Jul 01 14:39:47.387077 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.387067    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"]
Jul 01 14:39:47.387799 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.387749    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzg67\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.389048 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.389028    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-zgshx"]
Jul 01 14:39:47.390447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.390424    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 01 14:39:47.395130 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.395107    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"]
Jul 01 14:39:47.424919 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.424880    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.425073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.424928    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.425073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.424953    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-snapshots\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.425073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.424996    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-service-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.425073 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425026    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.425073 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.425049    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:39:47.425325 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.425136    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:47.925113656 +0000 UTC m=+36.641498416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:39:47.425325 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.425280    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:47.925262218 +0000 UTC m=+36.641646971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:39:47.425325 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425055    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.425483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425331    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.425483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425367    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.425483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425398    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrp27\" (UniqueName: \"kubernetes.io/projected/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-kube-api-access-nrp27\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.425483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425424    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s769\" (UniqueName: \"kubernetes.io/projected/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-kube-api-access-5s769\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.425483 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425468    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dh55\" (UniqueName: \"kubernetes.io/projected/70317fe5-e0e4-43f4-a19a-483c4b33fd40-kube-api-access-4dh55\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425496    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/357a4182-6b1f-4064-9a16-d280230a9061-cert\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.425514    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425522    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46mhr\" (UniqueName: \"kubernetes.io/projected/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-kube-api-access-46mhr\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.425570    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:47.925555953 +0000 UTC m=+36.641940688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425588    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-stats-auth\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425633    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-tmp\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425654    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.425710 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425668    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-snapshots\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.426111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425931    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-tmp\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.426111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.425657    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a922e63-f7a9-4190-9acc-20b9c68974cd-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.426111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426088    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lc4\" (UniqueName: \"kubernetes.io/projected/357a4182-6b1f-4064-9a16-d280230a9061-kube-api-access-56lc4\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426116    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426158    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv4g4\" (UniqueName: \"kubernetes.io/projected/2ca78877-45e9-427b-9ca5-ead578a8d572-kube-api-access-wv4g4\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426189    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-config\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426244    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mq4d\" (UniqueName: \"kubernetes.io/projected/4a922e63-f7a9-4190-9acc-20b9c68974cd-kube-api-access-8mq4d\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426265    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j25t\" (UniqueName: \"kubernetes.io/projected/ef110078-ba7d-4469-9fc8-3ae18fa17f51-kube-api-access-4j25t\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426282    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn5th\" (UniqueName: \"kubernetes.io/projected/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-kube-api-access-pn5th\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426306    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-serving-cert\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426337    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef110078-ba7d-4469-9fc8-3ae18fa17f51-serving-cert\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426357    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a922e63-f7a9-4190-9acc-20b9c68974cd-config\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426373    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca78877-45e9-427b-9ca5-ead578a8d572-config-volume\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426394    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.426418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426412    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ca78877-45e9-427b-9ca5-ead578a8d572-tmp-dir\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426445    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2kd\" (UniqueName: \"kubernetes.io/projected/b9de1d7e-87fd-4857-997c-51493ced5359-kube-api-access-hl2kd\") pod \"network-check-source-56499946bc-zgshx\" (UID: \"b9de1d7e-87fd-4857-997c-51493ced5359\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426460    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426459    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-service-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426494    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-default-certificate\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426516    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.427040 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.426533    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdx9s\" (UniqueName: \"kubernetes.io/projected/d309b969-9fb4-412d-b9c9-b793a68700c0-kube-api-access-tdx9s\") pod \"volume-data-source-validator-69d9f679db-f4vss\" (UID: \"d309b969-9fb4-412d-b9c9-b793a68700c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"
Jul 01 14:39:47.427531 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.427508    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a922e63-f7a9-4190-9acc-20b9c68974cd-config\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.427620 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.427602    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:47.427834 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.427810    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:47.927783437 +0000 UTC m=+36.644168173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:47.428006 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.427982    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.428692 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.428672    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.428864 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.428820    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.429355 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.429229    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-stats-auth\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.429757 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.429733    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-default-certificate\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.430047 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.430015    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a922e63-f7a9-4190-9acc-20b9c68974cd-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.430613 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.430595    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-serving-cert\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.463229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.462763    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s769\" (UniqueName: \"kubernetes.io/projected/61c6cdfc-f872-43a1-ace1-ee78a7c8b5af-kube-api-access-5s769\") pod \"kube-storage-version-migrator-operator-79964ff6f9-r8q4x\" (UID: \"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.466772 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.466692    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrp27\" (UniqueName: \"kubernetes.io/projected/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-kube-api-access-nrp27\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.469875 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.469796    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn5th\" (UniqueName: \"kubernetes.io/projected/b2b0c7e8-e90e-4093-ae0f-82e75413c8cf-kube-api-access-pn5th\") pod \"insights-operator-5f67b55555-7qj2r\" (UID: \"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf\") " pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.470376 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.469922    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mq4d\" (UniqueName: \"kubernetes.io/projected/4a922e63-f7a9-4190-9acc-20b9c68974cd-kube-api-access-8mq4d\") pod \"service-ca-operator-5d7b7dc95c-7d96s\" (UID: \"4a922e63-f7a9-4190-9acc-20b9c68974cd\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.470926 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.470897    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdx9s\" (UniqueName: \"kubernetes.io/projected/d309b969-9fb4-412d-b9c9-b793a68700c0-kube-api-access-tdx9s\") pod \"volume-data-source-validator-69d9f679db-f4vss\" (UID: \"d309b969-9fb4-412d-b9c9-b793a68700c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"
Jul 01 14:39:47.471027 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.470933    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46mhr\" (UniqueName: \"kubernetes.io/projected/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-kube-api-access-46mhr\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.471365 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.471319    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dh55\" (UniqueName: \"kubernetes.io/projected/70317fe5-e0e4-43f4-a19a-483c4b33fd40-kube-api-access-4dh55\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.500138 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.500112    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-7qj2r"
Jul 01 14:39:47.508911 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.508884    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"
Jul 01 14:39:47.515685 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.515656    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"
Jul 01 14:39:47.527627 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527604    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0cf4325c-d583-4906-98e8-ac0332a96536-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.527764 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527637    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2kd\" (UniqueName: \"kubernetes.io/projected/b9de1d7e-87fd-4857-997c-51493ced5359-kube-api-access-hl2kd\") pod \"network-check-source-56499946bc-zgshx\" (UID: \"b9de1d7e-87fd-4857-997c-51493ced5359\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx"
Jul 01 14:39:47.527764 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527657    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.527898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527783    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/357a4182-6b1f-4064-9a16-d280230a9061-cert\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.527898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527832    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lc4\" (UniqueName: \"kubernetes.io/projected/357a4182-6b1f-4064-9a16-d280230a9061-kube-api-access-56lc4\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.527898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527878    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.527898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527895    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4g4\" (UniqueName: \"kubernetes.io/projected/2ca78877-45e9-427b-9ca5-ead578a8d572-kube-api-access-wv4g4\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.528062 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.527918    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-config\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.528062 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.528008    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:47.528144 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.528086    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.028066215 +0000 UTC m=+36.744450957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:47.528203 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528008    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j25t\" (UniqueName: \"kubernetes.io/projected/ef110078-ba7d-4469-9fc8-3ae18fa17f51-kube-api-access-4j25t\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.528255 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528215    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef110078-ba7d-4469-9fc8-3ae18fa17f51-serving-cert\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.528307 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528259    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca78877-45e9-427b-9ca5-ead578a8d572-config-volume\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.528307 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528298    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.528401 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528328    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ca78877-45e9-427b-9ca5-ead578a8d572-tmp-dir\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.528488 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528461    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.528695 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528652    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef110078-ba7d-4469-9fc8-3ae18fa17f51-config\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.528917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.528895    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ca78877-45e9-427b-9ca5-ead578a8d572-config-volume\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.529146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.529128    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ca78877-45e9-427b-9ca5-ead578a8d572-tmp-dir\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.531189 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.531161    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef110078-ba7d-4469-9fc8-3ae18fa17f51-serving-cert\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.531295 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.531274    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/357a4182-6b1f-4064-9a16-d280230a9061-cert\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.540979 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.540959    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"
Jul 01 14:39:47.552185 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.552161    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv4g4\" (UniqueName: \"kubernetes.io/projected/2ca78877-45e9-427b-9ca5-ead578a8d572-kube-api-access-wv4g4\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:47.552263 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.552194    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2kd\" (UniqueName: \"kubernetes.io/projected/b9de1d7e-87fd-4857-997c-51493ced5359-kube-api-access-hl2kd\") pod \"network-check-source-56499946bc-zgshx\" (UID: \"b9de1d7e-87fd-4857-997c-51493ced5359\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx"
Jul 01 14:39:47.553170 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.553146    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lc4\" (UniqueName: \"kubernetes.io/projected/357a4182-6b1f-4064-9a16-d280230a9061-kube-api-access-56lc4\") pod \"ingress-canary-qhzkw\" (UID: \"357a4182-6b1f-4064-9a16-d280230a9061\") " pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.553497 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.553479    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j25t\" (UniqueName: \"kubernetes.io/projected/ef110078-ba7d-4469-9fc8-3ae18fa17f51-kube-api-access-4j25t\") pod \"console-operator-7fdbdf9c99-z8bwx\" (UID: \"ef110078-ba7d-4469-9fc8-3ae18fa17f51\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.590663 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.590588    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:47.629061 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.629028    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.629229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.629086    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0cf4325c-d583-4906-98e8-ac0332a96536-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.629229 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.629186    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:39:47.629348 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.629294    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.129277909 +0000 UTC m=+36.845662652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:39:47.629820 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.629801    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0cf4325c-d583-4906-98e8-ac0332a96536-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:47.669014 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.668981    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:39:47.683495 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.683468    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx"
Jul 01 14:39:47.701595 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.701566    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qhzkw"
Jul 01 14:39:47.712637 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.712604    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:47.718830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.718627    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:39:47.718830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.718645    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:39:47.718830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.718784    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:39:47.722523 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.722500    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpckn\""
Jul 01 14:39:47.722625 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.722500    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9fbg\""
Jul 01 14:39:47.722625 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.722584    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 14:39:47.722755 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.722731    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 14:39:47.830750 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.830717    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:47.830940 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.830905    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:47.830940 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.830921    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:39:47.831049 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.830983    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.83096787 +0000 UTC m=+37.547352619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:39:47.904666 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.904588    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:47.932015 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.931977    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:47.932015 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.932015    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.932053    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.932085    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932151    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932153    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932184    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932220    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.932203862 +0000 UTC m=+37.648588597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932243    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.932236442 +0000 UTC m=+37.648621176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:39:47.932252 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932254    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.932248189 +0000 UTC m=+37.648632923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:47.932517 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:47.932268    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:48.932259651 +0000 UTC m=+37.648644385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:39:47.975163 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:47.975124    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:48.032893 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.032835    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:48.033087 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.032997    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:48.033087 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.033071    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:49.033053157 +0000 UTC m=+37.749437906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:48.073904 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.073871    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:48.133721 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.133677    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:48.133930 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.133870    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:39:48.133989 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.133939    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:49.133922387 +0000 UTC m=+37.850307122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:39:48.182769 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.182735    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:48.840015 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.839981    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:48.840195 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.840145    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:48.840195 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.840164    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:39:48.840302 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.840231    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:50.840211897 +0000 UTC m=+39.556596641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:39:48.940963 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.940927    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:48.941145 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941083    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:39:48.941145 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.941089    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:48.941145 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.941118    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:48.941145 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941146    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:50.941127855 +0000 UTC m=+39.657512610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:39:48.941363 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941178    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:39:48.941363 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941210    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:50.941199966 +0000 UTC m=+39.657584705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:39:48.941363 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941217    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:48.941363 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941283    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:50.941264403 +0000 UTC m=+39.657649137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:48.941363 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.941316    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:48.941525 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:48.941419    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:50.941407292 +0000 UTC m=+39.657792039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:39:48.960756 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:48.960727    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:49.042235 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.042204    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:49.042947 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:49.042476    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:49.042947 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:49.042543    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:51.042526224 +0000 UTC m=+39.758910960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:49.058198 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.058147    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:49.145231 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.144294    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:49.145231 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:49.144611    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:39:49.145231 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:49.144685    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:51.144662736 +0000 UTC m=+39.861047510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:39:49.251735 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.251463    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x"]
Jul 01 14:39:49.280915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.280819    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-zgshx"]
Jul 01 14:39:49.285174 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:49.285144    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9de1d7e_87fd_4857_997c_51493ced5359.slice/crio-156f33699c63dced4301c12afc4c024f0b5e5abb2ee1d304cea8a9e222dd3e2a WatchSource:0}: Error finding container 156f33699c63dced4301c12afc4c024f0b5e5abb2ee1d304cea8a9e222dd3e2a: Status 404 returned error can't find the container with id 156f33699c63dced4301c12afc4c024f0b5e5abb2ee1d304cea8a9e222dd3e2a
Jul 01 14:39:49.291455 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.291435    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"]
Jul 01 14:39:49.295885 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:49.295814    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef110078_ba7d_4469_9fc8_3ae18fa17f51.slice/crio-aadfbd4288ef0d4b6bd8fa192d22ba998c52d4bcf7803da86524cc2e39be760a WatchSource:0}: Error finding container aadfbd4288ef0d4b6bd8fa192d22ba998c52d4bcf7803da86524cc2e39be760a: Status 404 returned error can't find the container with id aadfbd4288ef0d4b6bd8fa192d22ba998c52d4bcf7803da86524cc2e39be760a
Jul 01 14:39:49.304149 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.304128    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss"]
Jul 01 14:39:49.309083 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.308342    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-7qj2r"]
Jul 01 14:39:49.313881 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.313854    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s"]
Jul 01 14:39:49.316049 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.316027    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qhzkw"]
Jul 01 14:39:49.323758 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:39:49.323684    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd309b969_9fb4_412d_b9c9_b793a68700c0.slice/crio-393517e78d9fe19dc95662da8c2f66e2f4cad92cae6d3afc1b1fef62d095043f WatchSource:0}: Error finding container 393517e78d9fe19dc95662da8c2f66e2f4cad92cae6d3afc1b1fef62d095043f: Status 404 returned error can't find the container with id 393517e78d9fe19dc95662da8c2f66e2f4cad92cae6d3afc1b1fef62d095043f
Jul 01 14:39:49.907959 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.907912    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" event={"ID":"ef110078-ba7d-4469-9fc8-3ae18fa17f51","Type":"ContainerStarted","Data":"aadfbd4288ef0d4b6bd8fa192d22ba998c52d4bcf7803da86524cc2e39be760a"}
Jul 01 14:39:49.909906 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.909832    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-7qj2r" event={"ID":"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf","Type":"ContainerStarted","Data":"b0e388f7ef0d9211ef268dd1734b9b27bca0ee2fa3cb431935465b05a6eff5b4"}
Jul 01 14:39:49.912790 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.912759    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s" event={"ID":"4a922e63-f7a9-4190-9acc-20b9c68974cd","Type":"ContainerStarted","Data":"eff6fc8b174e6e9452d00ab47f00943cb8b4e0c44a399aa38898146360bf44c3"}
Jul 01 14:39:49.914570 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.914520    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss" event={"ID":"d309b969-9fb4-412d-b9c9-b793a68700c0","Type":"ContainerStarted","Data":"393517e78d9fe19dc95662da8c2f66e2f4cad92cae6d3afc1b1fef62d095043f"}
Jul 01 14:39:49.916192 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.916135    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qhzkw" event={"ID":"357a4182-6b1f-4064-9a16-d280230a9061","Type":"ContainerStarted","Data":"c2cfa36efb087107203a0c69756aff27dc936e00aa3fa58d199abe13cf37934a"}
Jul 01 14:39:49.917718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.917693    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x" event={"ID":"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af","Type":"ContainerStarted","Data":"5fd92225b450e7c189d3d01e48c0e8e2d59f67ca5f0f7886c9bf6dc080f7a925"}
Jul 01 14:39:49.923646 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.923620    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="2ba48c5f3fe2096d025c2ee9445090aba1c054b3e3d5f76659ed6aa32fd101dd" exitCode=0
Jul 01 14:39:49.923744 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.923702    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"2ba48c5f3fe2096d025c2ee9445090aba1c054b3e3d5f76659ed6aa32fd101dd"}
Jul 01 14:39:49.925891 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:49.925868    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx" event={"ID":"b9de1d7e-87fd-4857-997c-51493ced5359","Type":"ContainerStarted","Data":"156f33699c63dced4301c12afc4c024f0b5e5abb2ee1d304cea8a9e222dd3e2a"}
Jul 01 14:39:50.392273 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.392167    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:50.863557 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.862769    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:50.863557 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.863115    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:50.863557 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.863133    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:39:50.863557 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.863195    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:54.863175513 +0000 UTC m=+43.579560254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:39:50.938008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.937971    2541 generic.go:356] "Generic (PLEG): container finished" podID="bb43336f-c51e-4d6f-8708-89f7e9713860" containerID="f892e201887c203f561bb69bd42557168a050f6b85d22418d40879ceb367ebca" exitCode=0
Jul 01 14:39:50.938183 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.938048    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerDied","Data":"f892e201887c203f561bb69bd42557168a050f6b85d22418d40879ceb367ebca"}
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.963686    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.963737    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.963773    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:50.963810    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.963975    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964037    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:54.964017037 +0000 UTC m=+43.680401774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964100    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964133    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:54.964122083 +0000 UTC m=+43.680506824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964187    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964218    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:54.964206855 +0000 UTC m=+43.680591591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:39:50.964629 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:50.964282    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:54.964271775 +0000 UTC m=+43.680656511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:39:51.065540 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:51.065475    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:51.066147 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:51.066021    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:51.066147 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:51.066093    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:55.06607304 +0000 UTC m=+43.782457775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:51.166255 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:51.166218    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:51.166540 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:51.166525    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:39:51.166615 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:51.166592    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:55.166574883 +0000 UTC m=+43.882959621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:39:51.945383 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:51.945339    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" event={"ID":"bb43336f-c51e-4d6f-8708-89f7e9713860","Type":"ContainerStarted","Data":"05127296b790a32ecf1c03201f0a6abf700a8efe77f4699d39d27ba0fc83e238"}
Jul 01 14:39:51.976217 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:51.976126    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fqlzc" podStartSLOduration=10.141043602 podStartE2EDuration="40.976109138s" totalImagesPullingTime="30.835065536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:51.975159064 +0000 UTC m=+40.691543821" watchObservedRunningTime="2026-07-01 14:39:51.976109138 +0000 UTC m=+40.692493894"
Jul 01 14:39:52.103585 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:52.103549    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:52.211009 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:52.210941    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:52.721502 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:52.721469    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:52.819126 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:52.819092    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:54.905859 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:54.905800    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:39:54.906475 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:54.906012    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:54.906475 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:54.906032    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:39:54.906475 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:54.906090    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:02.906072235 +0000 UTC m=+51.622456975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:39:55.006534 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.006486    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:39:55.006716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.006541    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:39:55.006716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.006576    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:55.006716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.006609    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:39:55.006716 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006659    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:55.006716 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006688    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:39:55.006938 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006733    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.006711626 +0000 UTC m=+51.723096377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:39:55.006938 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006753    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.006742567 +0000 UTC m=+51.723127314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:39:55.006938 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006766    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.006758629 +0000 UTC m=+51.723143368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:39:55.006938 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006801    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:39:55.006938 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.006867    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.006855616 +0000 UTC m=+51.723240366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:39:55.107344 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.107297    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:39:55.107511 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.107474    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:55.107555 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.107536    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.107515403 +0000 UTC m=+51.823900138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:55.208530 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:55.208491    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:39:55.208696 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.208664    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:39:55.208751 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:55.208741    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.208717542 +0000 UTC m=+51.925102292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:39:57.963134 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.963092    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-7qj2r" event={"ID":"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf","Type":"ContainerStarted","Data":"4b1c050dcba855506234a47d5598695cf708014a8f858be148c68c3ef209a9de"}
Jul 01 14:39:57.965023 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.964997    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s" event={"ID":"4a922e63-f7a9-4190-9acc-20b9c68974cd","Type":"ContainerStarted","Data":"3227d5e0c178e6364559827d4d938f057d2444d57990e8993196f51f160087ed"}
Jul 01 14:39:57.966722 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.966695    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss" event={"ID":"d309b969-9fb4-412d-b9c9-b793a68700c0","Type":"ContainerStarted","Data":"c66aa9b1124982799506feea75ea22a3a5dfdb9ea7a3a63d259bd1f8eba848a6"}
Jul 01 14:39:57.969038 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.969016    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qhzkw" event={"ID":"357a4182-6b1f-4064-9a16-d280230a9061","Type":"ContainerStarted","Data":"291c04123ff96d1b04626e4ba5cf20a08f397633c3c5eae7e0fe4073d2b19f31"}
Jul 01 14:39:57.970562 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.970538    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x" event={"ID":"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af","Type":"ContainerStarted","Data":"a6260cb76284615df6dbcbda57447a6af7952357b70daa62916d5c3176656216"}
Jul 01 14:39:57.972346 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.972324    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx" event={"ID":"b9de1d7e-87fd-4857-997c-51493ced5359","Type":"ContainerStarted","Data":"22fe441a41bb06eeb9ccddd8c8411cfe70b5ae31e49df07236f82dd22ee91491"}
Jul 01 14:39:57.974474 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.974456    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/0.log"
Jul 01 14:39:57.974568 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.974488    2541 generic.go:356] "Generic (PLEG): container finished" podID="ef110078-ba7d-4469-9fc8-3ae18fa17f51" containerID="0785d19bdba874c75966b2ce7279e132c660f0f7c85676a9f940492f7df6b556" exitCode=255
Jul 01 14:39:57.974568 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.974521    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" event={"ID":"ef110078-ba7d-4469-9fc8-3ae18fa17f51","Type":"ContainerDied","Data":"0785d19bdba874c75966b2ce7279e132c660f0f7c85676a9f940492f7df6b556"}
Jul 01 14:39:57.974791 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.974776    2541 scope.go:117] "RemoveContainer" containerID="0785d19bdba874c75966b2ce7279e132c660f0f7c85676a9f940492f7df6b556"
Jul 01 14:39:57.985246 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:57.985190    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-7qj2r" podStartSLOduration=24.205096332 podStartE2EDuration="31.98517243s" totalImagesPullingTime="7.780076098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:57.984903191 +0000 UTC m=+46.701287950" watchObservedRunningTime="2026-07-01 14:39:57.98517243 +0000 UTC m=+46.701557191"
Jul 01 14:39:58.004903 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.004830    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-zgshx" podStartSLOduration=22.185326536 podStartE2EDuration="30.004808002s" totalImagesPullingTime="7.819481466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:58.002502732 +0000 UTC m=+46.718887490" watchObservedRunningTime="2026-07-01 14:39:58.004808002 +0000 UTC m=+46.721192761"
Jul 01 14:39:58.053503 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.053439    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x" podStartSLOduration=23.858536344 podStartE2EDuration="32.053415166s" totalImagesPullingTime="8.194878822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:58.051617996 +0000 UTC m=+46.768002751" watchObservedRunningTime="2026-07-01 14:39:58.053415166 +0000 UTC m=+46.769799925"
Jul 01 14:39:58.054229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.054197    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s" podStartSLOduration=25.038201404 podStartE2EDuration="32.054187067s" totalImagesPullingTime="7.015985663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:58.028283144 +0000 UTC m=+46.744667902" watchObservedRunningTime="2026-07-01 14:39:58.054187067 +0000 UTC m=+46.770571828"
Jul 01 14:39:58.083093 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.083035    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-f4vss" podStartSLOduration=24.299350311 podStartE2EDuration="32.083016187s" totalImagesPullingTime="7.783665876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:58.080228287 +0000 UTC m=+46.796613045" watchObservedRunningTime="2026-07-01 14:39:58.083016187 +0000 UTC m=+46.799400945"
Jul 01 14:39:58.167409 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.167364    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qhzkw" podStartSLOduration=3.385635867 podStartE2EDuration="11.167347279s" totalImagesPullingTime="7.781711412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:58.166867055 +0000 UTC m=+46.883251814" watchObservedRunningTime="2026-07-01 14:39:58.167347279 +0000 UTC m=+46.883732036"
Jul 01 14:39:58.980700 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.980667    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:39:58.981161 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.981110    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/0.log"
Jul 01 14:39:58.981161 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.981139    2541 generic.go:356] "Generic (PLEG): container finished" podID="ef110078-ba7d-4469-9fc8-3ae18fa17f51" containerID="194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3" exitCode=255
Jul 01 14:39:58.981269 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.981212    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" event={"ID":"ef110078-ba7d-4469-9fc8-3ae18fa17f51","Type":"ContainerDied","Data":"194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3"}
Jul 01 14:39:58.981334 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.981294    2541 scope.go:117] "RemoveContainer" containerID="0785d19bdba874c75966b2ce7279e132c660f0f7c85676a9f940492f7df6b556"
Jul 01 14:39:58.981537 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:58.981516    2541 scope.go:117] "RemoveContainer" containerID="194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3"
Jul 01 14:39:58.981748 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:58.981721    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8bwx_openshift-console-operator(ef110078-ba7d-4469-9fc8-3ae18fa17f51)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" podUID="ef110078-ba7d-4469-9fc8-3ae18fa17f51"
Jul 01 14:39:59.416224 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.416193    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnf6"]
Jul 01 14:39:59.444789 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.444761    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnf6"]
Jul 01 14:39:59.444948 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.444894    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.448127 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.448104    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 14:39:59.448270 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.448225    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 14:39:59.448595 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.448577    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wjs5n\""
Jul 01 14:39:59.548964 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.548930    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqcvp\" (UniqueName: \"kubernetes.io/projected/77a11177-c341-4312-be30-67977ad182ea-kube-api-access-cqcvp\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.549118 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.548973    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.549118 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.549050    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a11177-c341-4312-be30-67977ad182ea-data-volume\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.549198 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.549123    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77a11177-c341-4312-be30-67977ad182ea-crio-socket\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.549233 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.549221    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650485 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650452    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650646 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650575    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqcvp\" (UniqueName: \"kubernetes.io/projected/77a11177-c341-4312-be30-67977ad182ea-kube-api-access-cqcvp\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650646 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:59.650592    2541 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:39:59.650646 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650608    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650759 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:59.650667    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:00.150647932 +0000 UTC m=+48.867032688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : configmap "kube-rbac-proxy" not found
Jul 01 14:39:59.650759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650686    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a11177-c341-4312-be30-67977ad182ea-data-volume\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650759 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:59.650735    2541 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:39:59.650759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650754    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77a11177-c341-4312-be30-67977ad182ea-crio-socket\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.650969 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:59.650786    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:00.150769628 +0000 UTC m=+48.867154369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:39:59.650969 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.650815    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77a11177-c341-4312-be30-67977ad182ea-crio-socket\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.651162 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.651146    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a11177-c341-4312-be30-67977ad182ea-data-volume\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.667197 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.667140    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqcvp\" (UniqueName: \"kubernetes.io/projected/77a11177-c341-4312-be30-67977ad182ea-kube-api-access-cqcvp\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:39:59.955019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.954923    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:39:59.985397 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.985370    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:39:59.985802 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:39:59.985752    2541 scope.go:117] "RemoveContainer" containerID="194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3"
Jul 01 14:39:59.985957 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:39:59.985937    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8bwx_openshift-console-operator(ef110078-ba7d-4469-9fc8-3ae18fa17f51)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" podUID="ef110078-ba7d-4469-9fc8-3ae18fa17f51"
Jul 01 14:40:00.034587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.034555    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:00.102367 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.102333    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:00.155661 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.155621    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:00.155827 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.155761    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:00.155827 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:00.155774    2541 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:00.155920 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:00.155835    2541 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:00.155920 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:00.155851    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.155821787 +0000 UTC m=+49.872206522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:00.155920 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:00.155906    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.155890184 +0000 UTC m=+49.872274918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:00.557637 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.557601    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:00.660289 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.660252    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:00.823730 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:00.822117    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rb9zw_1cb87f50-27ec-4332-bb8d-5f33ffa8fa45/dns-node-resolver/0.log"
Jul 01 14:40:01.165066 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.165023    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:01.165545 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.165211    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:01.165545 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:01.165218    2541 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:01.165545 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:01.165376    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.165356291 +0000 UTC m=+51.881741030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:01.165545 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:01.165261    2541 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:01.165545 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:01.165456    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.165437548 +0000 UTC m=+51.881822288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:01.271116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.271078    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:01.466966 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.466880    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:01.565946 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.565914    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:01.710953 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.710920    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:01.821554 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:01.821474    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jjvg_31c63ed0-9e3f-4e22-8844-f28c0d5b9425/node-ca/0.log"
Jul 01 14:40:02.333980 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:02.333951    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:02.772558 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:02.772524    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:02.821406 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:02.821378    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qhzkw_357a4182-6b1f-4064-9a16-d280230a9061/serve-healthcheck-canary/0.log"
Jul 01 14:40:02.868456 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:02.868424    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:02.980774 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:02.980742    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:02.980970 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:02.980926    2541 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:40:02.980970 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:02.980943    2541 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd4cb4944-s82kp: secret "image-registry-tls" not found
Jul 01 14:40:02.981042 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:02.980997    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls podName:380ab168-967e-4928-b415-0467d4be0235 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:18.980980533 +0000 UTC m=+67.697365272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls") pod "image-registry-cd4cb4944-s82kp" (UID: "380ab168-967e-4928-b415-0467d4be0235") : secret "image-registry-tls" not found
Jul 01 14:40:03.081308 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.081215    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:40:03.081308 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.081260    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:40:03.081308 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.081282    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.081317    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081380    2541 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081384    2541 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081405    2541 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081441    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls podName:309e9e23-cd33-4e52-a9d9-6765a8cdc9d5 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.08142551 +0000 UTC m=+67.797810245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8klb9" (UID: "309e9e23-cd33-4e52-a9d9-6765a8cdc9d5") : secret "samples-operator-tls" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081456    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls podName:0858db31-5603-4785-b3ba-6dc0a6d3e7d7 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.081449659 +0000 UTC m=+67.797834394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-klcwt" (UID: "0858db31-5603-4785-b3ba-6dc0a6d3e7d7") : secret "cluster-monitoring-operator-tls" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081467    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.081461513 +0000 UTC m=+67.797846248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : secret "router-metrics-certs-default" not found
Jul 01 14:40:03.081526 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.081476    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle podName:70317fe5-e0e4-43f4-a19a-483c4b33fd40 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.081471578 +0000 UTC m=+67.797856313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle") pod "router-default-7b66db9b69-6qgkk" (UID: "70317fe5-e0e4-43f4-a19a-483c4b33fd40") : configmap references non-existent config key: service-ca.crt
Jul 01 14:40:03.182605 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.182562    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:03.182811 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.182692    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:03.182811 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182719    2541 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:40:03.182811 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182780    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls podName:2ca78877-45e9-427b-9ca5-ead578a8d572 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.182762946 +0000 UTC m=+67.899147703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls") pod "dns-default-nlnd6" (UID: "2ca78877-45e9-427b-9ca5-ead578a8d572") : secret "dns-default-metrics-tls" not found
Jul 01 14:40:03.182811 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.182808    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:03.183049 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182818    2541 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:03.183049 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182889    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:07.182875846 +0000 UTC m=+55.899260580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:03.183049 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182927    2541 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:03.183049 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.182960    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:07.182952864 +0000 UTC m=+55.899337600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:03.283574 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.283493    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:40:03.283750 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.283651    2541 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 14:40:03.283750 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:03.283717    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert podName:0cf4325c-d583-4906-98e8-ac0332a96536 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:19.283701784 +0000 UTC m=+68.000086520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kwj9r" (UID: "0cf4325c-d583-4906-98e8-ac0332a96536") : secret "networking-console-plugin-cert" not found
Jul 01 14:40:03.467237 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:03.467200    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:04.569771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:04.569736    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:06.848157 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:06.848121    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:06.989556 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:06.989525    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:07.219103 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.219070    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:07.219296 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.219182    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:07.219296 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:07.219223    2541 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:07.219296 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:07.219294    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls podName:77a11177-c341-4312-be30-67977ad182ea nodeName:}" failed. No retries permitted until 2026-07-01 14:40:15.219278747 +0000 UTC m=+63.935663481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6gnf6" (UID: "77a11177-c341-4312-be30-67977ad182ea") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:07.219652 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.219634    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77a11177-c341-4312-be30-67977ad182ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:07.669469 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.669437    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:40:07.669636 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.669525    2541 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:40:07.669907 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:07.669895    2541 scope.go:117] "RemoveContainer" containerID="194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3"
Jul 01 14:40:07.670057 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:07.670039    2541 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8bwx_openshift-console-operator(ef110078-ba7d-4469-9fc8-3ae18fa17f51)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" podUID="ef110078-ba7d-4469-9fc8-3ae18fa17f51"
Jul 01 14:40:09.803290 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:09.803255    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:09.902428 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:09.902396    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:09.989081 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:09.989048    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:10.160013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:10.159985    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:10.532859 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:10.532753    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:10.634890 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:10.634859    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:10.876716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:10.876645    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:11.527124 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:11.527092    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:11.593892 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:11.593861    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:11.950109 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:11.950080    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:12.144438 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:12.144404    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:12.744988 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:12.744955    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:12.970084 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:12.970049    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:13.114193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:13.114099    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:13.340894 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:13.340861    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:13.539965 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:13.539932    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:13.899667 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:13.899593    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh22p"
Jul 01 14:40:13.959024 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:13.958994    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:14.178297 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:14.178267    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:14.739658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:14.739623    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:15.292091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.292054    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:15.294393 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.294358    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77a11177-c341-4312-be30-67977ad182ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6gnf6\" (UID: \"77a11177-c341-4312-be30-67977ad182ea\") " pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:15.342629 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.342595    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:15.358707 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.358684    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wjs5n\""
Jul 01 14:40:15.366820 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.366803    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6gnf6"
Jul 01 14:40:15.497563 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.497539    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6gnf6"]
Jul 01 14:40:15.500156 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:15.500125    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a11177_c341_4312_be30_67977ad182ea.slice/crio-1db3ade60c3993e85cb99e040ff3f670a1f338abfda3e603d375d8d7fb21a27b WatchSource:0}: Error finding container 1db3ade60c3993e85cb99e040ff3f670a1f338abfda3e603d375d8d7fb21a27b: Status 404 returned error can't find the container with id 1db3ade60c3993e85cb99e040ff3f670a1f338abfda3e603d375d8d7fb21a27b
Jul 01 14:40:15.539422 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.539390    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:15.605621 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.605541    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:15.739335 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:15.739306    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:16.044158 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:16.044125    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnf6" event={"ID":"77a11177-c341-4312-be30-67977ad182ea","Type":"ContainerStarted","Data":"1db3ade60c3993e85cb99e040ff3f670a1f338abfda3e603d375d8d7fb21a27b"}
Jul 01 14:40:16.144983 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:16.144952    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:16.740740 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:16.740690    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:17.143944 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.143855    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:17.342272 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.342237    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:17.413687 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.413658    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:40:17.416714 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.416688    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 14:40:17.426598 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.426579    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3500fb9f-71cb-4c7d-a1f7-9d58da2185eb-metrics-certs\") pod \"network-metrics-daemon-p5s2b\" (UID: \"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb\") " pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:40:17.440486 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.440462    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9fbg\""
Jul 01 14:40:17.448401 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.448375    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5s2b"
Jul 01 14:40:17.514883 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.514786    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:40:17.515031 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.514943    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:40:17.517816 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.517784    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4lcq\" (UniqueName: \"kubernetes.io/projected/0c67366f-5b37-478a-911f-0d0fe521267b-kube-api-access-b4lcq\") pod \"network-check-target-dz29r\" (UID: \"0c67366f-5b37-478a-911f-0d0fe521267b\") " pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:40:17.518165 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.518147    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 14:40:17.528198 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.528172    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9143e78e-04cd-4a6b-8fd3-a25bf0aa0836-original-pull-secret\") pod \"global-pull-secret-syncer-mwx8j\" (UID: \"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836\") " pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:40:17.537054 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.537029    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:17.594279 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.594204    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p5s2b"]
Jul 01 14:40:17.598326 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:17.598289    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3500fb9f_71cb_4c7d_a1f7_9d58da2185eb.slice/crio-e68a6cffa988176dfe8d22d780c0b31b24d1dacd81c6958976da511f5a25538a WatchSource:0}: Error finding container e68a6cffa988176dfe8d22d780c0b31b24d1dacd81c6958976da511f5a25538a: Status 404 returned error can't find the container with id e68a6cffa988176dfe8d22d780c0b31b24d1dacd81c6958976da511f5a25538a
Jul 01 14:40:17.733890 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.733797    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpckn\""
Jul 01 14:40:17.741104 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.741079    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:40:17.743380 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.743362    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mwx8j"
Jul 01 14:40:17.908376 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.908288    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mwx8j"]
Jul 01 14:40:17.911689 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:17.911647    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9143e78e_04cd_4a6b_8fd3_a25bf0aa0836.slice/crio-22d24bab59e8d90c3bd5b33d42928d0d5d534a1531502951ebc16f21b8cd8f03 WatchSource:0}: Error finding container 22d24bab59e8d90c3bd5b33d42928d0d5d534a1531502951ebc16f21b8cd8f03: Status 404 returned error can't find the container with id 22d24bab59e8d90c3bd5b33d42928d0d5d534a1531502951ebc16f21b8cd8f03
Jul 01 14:40:17.930421 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:17.930397    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dz29r"]
Jul 01 14:40:17.935568 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:17.935541    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c67366f_5b37_478a_911f_0d0fe521267b.slice/crio-13a4abb5ccbee1fea8ae7f312892d6ee5cc8de1238e3c56981ae62d094815217 WatchSource:0}: Error finding container 13a4abb5ccbee1fea8ae7f312892d6ee5cc8de1238e3c56981ae62d094815217: Status 404 returned error can't find the container with id 13a4abb5ccbee1fea8ae7f312892d6ee5cc8de1238e3c56981ae62d094815217
Jul 01 14:40:18.054170 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.054137    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mwx8j" event={"ID":"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836","Type":"ContainerStarted","Data":"22d24bab59e8d90c3bd5b33d42928d0d5d534a1531502951ebc16f21b8cd8f03"}
Jul 01 14:40:18.055781 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.055718    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5s2b" event={"ID":"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb","Type":"ContainerStarted","Data":"e68a6cffa988176dfe8d22d780c0b31b24d1dacd81c6958976da511f5a25538a"}
Jul 01 14:40:18.058748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.057865    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dz29r" event={"ID":"0c67366f-5b37-478a-911f-0d0fe521267b","Type":"ContainerStarted","Data":"1237fe986cbe5c526320aac4a7318c6222411a2d32ddd53be9e8d2f6ebafbcbd"}
Jul 01 14:40:18.058748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.057907    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dz29r" event={"ID":"0c67366f-5b37-478a-911f-0d0fe521267b","Type":"ContainerStarted","Data":"13a4abb5ccbee1fea8ae7f312892d6ee5cc8de1238e3c56981ae62d094815217"}
Jul 01 14:40:18.058748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.058708    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:40:18.078858 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.077951    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dz29r" podStartSLOduration=66.077931273 podStartE2EDuration="1m6.077931273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:18.077805422 +0000 UTC m=+66.794190183" watchObservedRunningTime="2026-07-01 14:40:18.077931273 +0000 UTC m=+66.794316032"
Jul 01 14:40:18.719446 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:18.719408    2541 scope.go:117] "RemoveContainer" containerID="194888a8b72509c468e4dfa00b322c4b0b6ca6cf65489d5119eccfa0b09dabd3"
Jul 01 14:40:19.031680 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.031582    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:19.034467 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.034436    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"image-registry-cd4cb4944-s82kp\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") " pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:19.132981 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.132946    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:40:19.133181 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.132997    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:40:19.133181 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.133023    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:19.133181 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.133053    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:19.134216 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.134183    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70317fe5-e0e4-43f4-a19a-483c4b33fd40-service-ca-bundle\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:19.135791 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.135764    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70317fe5-e0e4-43f4-a19a-483c4b33fd40-metrics-certs\") pod \"router-default-7b66db9b69-6qgkk\" (UID: \"70317fe5-e0e4-43f4-a19a-483c4b33fd40\") " pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:19.136284 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.135959    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/309e9e23-cd33-4e52-a9d9-6765a8cdc9d5-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8klb9\" (UID: \"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:40:19.136956 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.136938    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0858db31-5603-4785-b3ba-6dc0a6d3e7d7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-klcwt\" (UID: \"0858db31-5603-4785-b3ba-6dc0a6d3e7d7\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:40:19.234531 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.234497    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:19.236789 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.236766    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ca78877-45e9-427b-9ca5-ead578a8d572-metrics-tls\") pod \"dns-default-nlnd6\" (UID: \"2ca78877-45e9-427b-9ca5-ead578a8d572\") " pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:19.261019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.260989    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45ph4\""
Jul 01 14:40:19.268797 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.268775    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:19.295788 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.295767    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4n8bh\""
Jul 01 14:40:19.303229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.303208    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"
Jul 01 14:40:19.329093 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.329069    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cdd9t\""
Jul 01 14:40:19.334894 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.334864    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:40:19.336219 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.336198    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"
Jul 01 14:40:19.337066 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.337047    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fbbrf\""
Jul 01 14:40:19.337265 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.337249    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0cf4325c-d583-4906-98e8-ac0332a96536-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kwj9r\" (UID: \"0cf4325c-d583-4906-98e8-ac0332a96536\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:40:19.345531 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.345512    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:19.426128 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.426098    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs5sl\""
Jul 01 14:40:19.434371 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.434339    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:19.513797 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.513764    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kn8p7\""
Jul 01 14:40:19.521144 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:19.521118    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"
Jul 01 14:40:21.238308 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.238233    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:40:21.253158 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.252708    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9"]
Jul 01 14:40:21.257389 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.257273    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b66db9b69-6qgkk"]
Jul 01 14:40:21.267076 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:21.265942    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70317fe5_e0e4_43f4_a19a_483c4b33fd40.slice/crio-27d26261ddf32e078d9e70844cf07b47f0971eed61bbe9f95048fd636e34cd15 WatchSource:0}: Error finding container 27d26261ddf32e078d9e70844cf07b47f0971eed61bbe9f95048fd636e34cd15: Status 404 returned error can't find the container with id 27d26261ddf32e078d9e70844cf07b47f0971eed61bbe9f95048fd636e34cd15
Jul 01 14:40:21.286490 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.286418    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt"]
Jul 01 14:40:21.295896 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.294725    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r"]
Jul 01 14:40:21.346208 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:21.346182    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlnd6"]
Jul 01 14:40:22.101103 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.101055    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnf6" event={"ID":"77a11177-c341-4312-be30-67977ad182ea","Type":"ContainerStarted","Data":"76ea9c70dbe5622282bdbbb814319916f830ee85e5f138333e234c2631d3789f"}
Jul 01 14:40:22.102863 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.102815    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt" event={"ID":"0858db31-5603-4785-b3ba-6dc0a6d3e7d7","Type":"ContainerStarted","Data":"4d63e58389f2c816a03f5a6b5e2e81a6481506793b850ecb49a8f106a6a1c21b"}
Jul 01 14:40:22.105397 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.104900    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b66db9b69-6qgkk" event={"ID":"70317fe5-e0e4-43f4-a19a-483c4b33fd40","Type":"ContainerStarted","Data":"41c805148dadf585f2faa0a15b4913adf45e7d7502ecf2a21440ad6be4cf388f"}
Jul 01 14:40:22.105397 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.104952    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b66db9b69-6qgkk" event={"ID":"70317fe5-e0e4-43f4-a19a-483c4b33fd40","Type":"ContainerStarted","Data":"27d26261ddf32e078d9e70844cf07b47f0971eed61bbe9f95048fd636e34cd15"}
Jul 01 14:40:22.107103 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.107075    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" event={"ID":"380ab168-967e-4928-b415-0467d4be0235","Type":"ContainerStarted","Data":"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"}
Jul 01 14:40:22.107206 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.107122    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" event={"ID":"380ab168-967e-4928-b415-0467d4be0235","Type":"ContainerStarted","Data":"451e915fbe9e6611aa1a395ee97c3d7f77577da09a7310fcef871867f42b656f"}
Jul 01 14:40:22.107578 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.107558    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:22.110076 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.110045    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5s2b" event={"ID":"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb","Type":"ContainerStarted","Data":"0171ed16d6f82cd2fd2498c52d950d9efa69195064488d478c3ea33b5a3a3600"}
Jul 01 14:40:22.110171 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.110099    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5s2b" event={"ID":"3500fb9f-71cb-4c7d-a1f7-9d58da2185eb","Type":"ContainerStarted","Data":"2cae6e642dc04d43f04e6d15ca8cb97bd72ab2856719f69ba6f080a21db6ec6a"}
Jul 01 14:40:22.111705 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.111680    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlnd6" event={"ID":"2ca78877-45e9-427b-9ca5-ead578a8d572","Type":"ContainerStarted","Data":"2d51f60368840f8ee451ee21030f80a6c06941fe0e0480ec0d1a7a786b4eb5f7"}
Jul 01 14:40:22.113002 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.112974    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r" event={"ID":"0cf4325c-d583-4906-98e8-ac0332a96536","Type":"ContainerStarted","Data":"dbc48ecb68c4f7242fee7a2ea4e6e246bb8dfc05b4d0478fee49bf43efeb89bc"}
Jul 01 14:40:22.114666 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.114635    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9" event={"ID":"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5","Type":"ContainerStarted","Data":"670e63f7aa3c245bee6560e03a347ecd1e2a097237ccc596d962a26050a64c2d"}
Jul 01 14:40:22.116736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.116703    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:40:22.116822 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.116742    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" event={"ID":"ef110078-ba7d-4469-9fc8-3ae18fa17f51","Type":"ContainerStarted","Data":"82af1d0a7891833223a14817b0a66005e99c91406a0af2c60e0f35b61fa807df"}
Jul 01 14:40:22.118618 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.118600    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:40:22.127303 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.127259    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b66db9b69-6qgkk" podStartSLOduration=56.127232225 podStartE2EDuration="56.127232225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:22.12636183 +0000 UTC m=+70.842746600" watchObservedRunningTime="2026-07-01 14:40:22.127232225 +0000 UTC m=+70.843616975"
Jul 01 14:40:22.145825 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.145745    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx" podStartSLOduration=48.336366988 podStartE2EDuration="56.145731865s" totalImagesPullingTime="7.809364877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:22.145553981 +0000 UTC m=+70.861938739" watchObservedRunningTime="2026-07-01 14:40:22.145731865 +0000 UTC m=+70.862116622"
Jul 01 14:40:22.164161 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.164107    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p5s2b" podStartSLOduration=67.780802189 podStartE2EDuration="1m11.164061965s" totalImagesPullingTime="3.383259776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:22.162801517 +0000 UTC m=+70.879186275" watchObservedRunningTime="2026-07-01 14:40:22.164061965 +0000 UTC m=+70.880446724"
Jul 01 14:40:22.183449 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.183394    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" podStartSLOduration=70.183374709 podStartE2EDuration="1m10.183374709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:22.182906188 +0000 UTC m=+70.899290947" watchObservedRunningTime="2026-07-01 14:40:22.183374709 +0000 UTC m=+70.899759468"
Jul 01 14:40:22.346363 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.346329    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:22.348813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.348787    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:22.561275 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.561245    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8bwx"
Jul 01 14:40:22.798992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.798960    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:22.905226 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:22.905138    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:23.121513 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.121259    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:23.122674 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.122647    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b66db9b69-6qgkk"
Jul 01 14:40:23.303634 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.303606    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:23.513618 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.513576    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:23.844434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.844398    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:23.926744 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:23.926708    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:24.824566 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:24.824530    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:24.918318 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:24.918289    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:25.009657 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.009635    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:25.138772 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.138399    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r" event={"ID":"0cf4325c-d583-4906-98e8-ac0332a96536","Type":"ContainerStarted","Data":"3e7919994247563723e9a7ba5bc05380362a9cedb4b4baac2f91d2839cd21bbb"}
Jul 01 14:40:25.141449 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.141375    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnf6" event={"ID":"77a11177-c341-4312-be30-67977ad182ea","Type":"ContainerStarted","Data":"732f87051866c603caad66b1ef4c3d89ec02628177c8bafc374184562bacc1bf"}
Jul 01 14:40:25.144769 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.144740    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt" event={"ID":"0858db31-5603-4785-b3ba-6dc0a6d3e7d7","Type":"ContainerStarted","Data":"b50fd1dc1623ddfb16a48995397781f344b91daf50fd1fd5950c22871daa77d4"}
Jul 01 14:40:25.186009 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.185693    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kwj9r" podStartSLOduration=48.487528427 podStartE2EDuration="52.185669879s" totalImagesPullingTime="3.698141452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:25.161982697 +0000 UTC m=+73.878367456" watchObservedRunningTime="2026-07-01 14:40:25.185669879 +0000 UTC m=+73.902054636"
Jul 01 14:40:25.187485 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.186426    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-klcwt" podStartSLOduration=55.492210601 podStartE2EDuration="59.186413005s" totalImagesPullingTime="3.694202404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:25.184826795 +0000 UTC m=+73.901211553" watchObservedRunningTime="2026-07-01 14:40:25.186413005 +0000 UTC m=+73.902797763"
Jul 01 14:40:25.419546 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.419509    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:25.519815 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.519726    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:25.635556 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:25.635517    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.050765 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.050732    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.151936 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.151895    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mwx8j" event={"ID":"9143e78e-04cd-4a6b-8fd3-a25bf0aa0836","Type":"ContainerStarted","Data":"9fbd1ae8bbdc74cb2b4ab7755c8ad4fa505007760e5e25aa948403491d93a3ff"}
Jul 01 14:40:26.154461 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.153946    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlnd6" event={"ID":"2ca78877-45e9-427b-9ca5-ead578a8d572","Type":"ContainerStarted","Data":"0476f58ffc3b72a294a3ffa9932335d7523a9808c3590f1119a6756e367931b7"}
Jul 01 14:40:26.154461 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.153997    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlnd6" event={"ID":"2ca78877-45e9-427b-9ca5-ead578a8d572","Type":"ContainerStarted","Data":"1f60e33303906d7a1da8723814206a5ad0ed3caa4113b3441ac8f75ca4e3f3a6"}
Jul 01 14:40:26.154461 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.154029    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:26.232375 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.232313    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mwx8j" podStartSLOduration=67.146729094 podStartE2EDuration="1m14.23229079s" totalImagesPullingTime="7.085561696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:26.229552378 +0000 UTC m=+74.945937136" watchObservedRunningTime="2026-07-01 14:40:26.23229079 +0000 UTC m=+74.948675528"
Jul 01 14:40:26.287095 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.286789    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nlnd6" podStartSLOduration=35.648531725 podStartE2EDuration="39.286768814s" totalImagesPullingTime="3.638237089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:26.284997021 +0000 UTC m=+75.001381778" watchObservedRunningTime="2026-07-01 14:40:26.286768814 +0000 UTC m=+75.003153570"
Jul 01 14:40:26.288283 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.288219    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.376428 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.376330    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.484265 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.484189    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.585639 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.585610    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:26.724662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:26.724628    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:29.167581 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:29.167548    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6gnf6" event={"ID":"77a11177-c341-4312-be30-67977ad182ea","Type":"ContainerStarted","Data":"f35a4f6c3e9d31b6cdfef9e49f72a8aa11276da7066abfbb1b8b463717518851"}
Jul 01 14:40:29.169251 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:29.169221    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9" event={"ID":"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5","Type":"ContainerStarted","Data":"c9676191820c2e40c67594a7568475aea0515fe5d52938a80d772a48ca131d7a"}
Jul 01 14:40:29.169441 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:29.169270    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9" event={"ID":"309e9e23-cd33-4e52-a9d9-6765a8cdc9d5","Type":"ContainerStarted","Data":"30cf0aeda3025631e52704f5ddae7606a65f23109e28fadbbf5e50a9c83f3dfa"}
Jul 01 14:40:29.199848 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:29.199782    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6gnf6" podStartSLOduration=17.451379252 podStartE2EDuration="30.199763527s" totalImagesPullingTime="12.748384275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:29.199125116 +0000 UTC m=+77.915509873" watchObservedRunningTime="2026-07-01 14:40:29.199763527 +0000 UTC m=+77.916148287"
Jul 01 14:40:29.250776 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:29.250718    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8klb9" podStartSLOduration=56.132987177 podStartE2EDuration="1m3.250702854s" totalImagesPullingTime="7.117715677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:29.249031819 +0000 UTC m=+77.965416576" watchObservedRunningTime="2026-07-01 14:40:29.250702854 +0000 UTC m=+77.967087611"
Jul 01 14:40:30.546443 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:30.546408    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:30.647326 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:30.647292    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:31.737502 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:31.737471    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:31.833113 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:31.833081    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:31.972409 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:31.972278    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:32.065470 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:32.065363    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:32.165159 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:32.165120    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:32.562496 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:32.562463    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:32.664748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:32.664709    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:33.736445 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:33.736416    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:35.786998 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:35.786964    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:35.910032 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:35.910000    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:36.161385 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:36.161361    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nlnd6"
Jul 01 14:40:36.508402 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:36.508311    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:39.060029 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:39.059997    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:39.164143 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:39.164112    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:39.359071 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:39.358997    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:40.468678 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:40.468648    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:40.569860 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:40.569801    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:41.070571 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.070538    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:41.256309 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.256274    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:41.320342 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.320307    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"]
Jul 01 14:40:41.323620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.323557    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.326519 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.326495    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 01 14:40:41.326651 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.326586    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dxdt8\""
Jul 01 14:40:41.327709 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.327678    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 01 14:40:41.327813 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.327765    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 14:40:41.346882 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.346834    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"]
Jul 01 14:40:41.371157 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.371128    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dqm7n"]
Jul 01 14:40:41.373382 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.373365    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.375775 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.375755    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tpkpl\""
Jul 01 14:40:41.376254 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.376233    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 14:40:41.376323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.376309    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 14:40:41.376942 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.376929    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 14:40:41.447430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.447386    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:41.514673 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514645    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514691    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514746    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-sys\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514790    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-wtmp\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514816    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514831    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-metrics-client-ca\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514893    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ngt4\" (UniqueName: \"kubernetes.io/projected/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-kube-api-access-4ngt4\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514914    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-root\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.514929    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.515006    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxxk8\" (UniqueName: \"kubernetes.io/projected/ee6ca283-3584-4392-af96-b83cf6a4b97a-kube-api-access-lxxk8\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.515038    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.515098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.515068    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-textfile\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.515477 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.515117    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.615830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615755    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-root\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.615830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615788    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.615830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615816    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxxk8\" (UniqueName: \"kubernetes.io/projected/ee6ca283-3584-4392-af96-b83cf6a4b97a-kube-api-access-lxxk8\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615824    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-root\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615863    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615972    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-textfile\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.615994    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616023    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616079    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.616139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616124    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-sys\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616160    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-wtmp\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616192    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616215    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-metrics-client-ca\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616243    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ngt4\" (UniqueName: \"kubernetes.io/projected/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-kube-api-access-4ngt4\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.616484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616416    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-textfile\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616907 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616550    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-wtmp\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.616907 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:41.616553    2541 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 01 14:40:41.616907 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616586    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.616907 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:40:41.616623    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls podName:ee6ca283-3584-4392-af96-b83cf6a4b97a nodeName:}" failed. No retries permitted until 2026-07-01 14:40:42.116601976 +0000 UTC m=+90.832986714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls") pod "node-exporter-dqm7n" (UID: "ee6ca283-3584-4392-af96-b83cf6a4b97a") : secret "node-exporter-tls" not found
Jul 01 14:40:41.616907 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.616658    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ca283-3584-4392-af96-b83cf6a4b97a-sys\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.617169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.617073    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-metrics-client-ca\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.617225 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.617206    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.618588 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.618564    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.619111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.619089    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.619216 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.619144    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.625991 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.625952    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ngt4\" (UniqueName: \"kubernetes.io/projected/383a8cfa-3e3e-4a5a-a5c8-91cca1468071-kube-api-access-4ngt4\") pod \"openshift-state-metrics-57f99c8b49-d5g9f\" (UID: \"383a8cfa-3e3e-4a5a-a5c8-91cca1468071\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.626369 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.626344    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxxk8\" (UniqueName: \"kubernetes.io/projected/ee6ca283-3584-4392-af96-b83cf6a4b97a-kube-api-access-lxxk8\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:41.633288 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.633256    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"
Jul 01 14:40:41.792477 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:41.792424    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f"]
Jul 01 14:40:41.795252 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:41.795223    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod383a8cfa_3e3e_4a5a_a5c8_91cca1468071.slice/crio-1a69381daca01209ce46db40ae12c595fb66cbf1dde7a078723db0765513c771 WatchSource:0}: Error finding container 1a69381daca01209ce46db40ae12c595fb66cbf1dde7a078723db0765513c771: Status 404 returned error can't find the container with id 1a69381daca01209ce46db40ae12c595fb66cbf1dde7a078723db0765513c771
Jul 01 14:40:42.120170 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.120130    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:42.122659 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.122628    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6ca283-3584-4392-af96-b83cf6a4b97a-node-exporter-tls\") pod \"node-exporter-dqm7n\" (UID: \"ee6ca283-3584-4392-af96-b83cf6a4b97a\") " pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:42.216981 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.216943    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f" event={"ID":"383a8cfa-3e3e-4a5a-a5c8-91cca1468071","Type":"ContainerStarted","Data":"1f1171a15e7035eb6f6b285ef6eff5e47cff55064514a3f664ad5aa02ac16762"}
Jul 01 14:40:42.217139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.217029    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f" event={"ID":"383a8cfa-3e3e-4a5a-a5c8-91cca1468071","Type":"ContainerStarted","Data":"30983fb3d2308b7b87f574b17a7a355c80441f9891a0dfb8a3805e4d9b5a2f77"}
Jul 01 14:40:42.217139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.217050    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f" event={"ID":"383a8cfa-3e3e-4a5a-a5c8-91cca1468071","Type":"ContainerStarted","Data":"1a69381daca01209ce46db40ae12c595fb66cbf1dde7a078723db0765513c771"}
Jul 01 14:40:42.282324 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.282252    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dqm7n"
Jul 01 14:40:42.293137 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:42.293110    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6ca283_3584_4392_af96_b83cf6a4b97a.slice/crio-25df08b693d436e6329132246f20c1b564d3329d821cb40f85cf4c3f5cee49d4 WatchSource:0}: Error finding container 25df08b693d436e6329132246f20c1b564d3329d821cb40f85cf4c3f5cee49d4: Status 404 returned error can't find the container with id 25df08b693d436e6329132246f20c1b564d3329d821cb40f85cf4c3f5cee49d4
Jul 01 14:40:42.486312 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.486233    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:40:42.489189 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.489170    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.492175 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.492134    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 14:40:42.492290 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.492238    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 14:40:42.492695 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.492678    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 14:40:42.492781 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.492727    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 14:40:42.492989 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.492974    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-thc26\""
Jul 01 14:40:42.493069 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.493005    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 14:40:42.493125 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.493079    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 14:40:42.493227 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.493211    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 14:40:42.493282 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.493267    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 14:40:42.495235 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.495204    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 14:40:42.567422 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.567390    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:40:42.623707 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623664    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623745    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623770    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623803    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623835    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623885    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.623915 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623905    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.623926    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.624027    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.624064    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6lf\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.624113    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.624139    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.624229 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.624169    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.724953 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.724917    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.724997    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725029    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725057    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725091    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725116    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725139    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725159    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725196    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725219    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6lf\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725268    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725290    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.725468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.725311    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.726283 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.726244    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.726712 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.726686    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.727720 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.727693    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.729975 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.729905    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.731496 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.731466    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.732123 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.731953    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.732123 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.732063    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.732123 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.732069    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.732507 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.732485    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.732794 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.732778    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.733336 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.733317    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.733884 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.733858    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.747154 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.747092    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6lf\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf\") pod \"alertmanager-main-0\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.799276 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.799242    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:40:42.995529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:42.995499    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:40:42.998244 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:42.998169    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f501d2f_2806_4eac_8dbb_423a6759d66d.slice/crio-e29aed816cfd2b625cd6a1b756bab7b36989622ee777e9b60a26144325d54a53 WatchSource:0}: Error finding container e29aed816cfd2b625cd6a1b756bab7b36989622ee777e9b60a26144325d54a53: Status 404 returned error can't find the container with id e29aed816cfd2b625cd6a1b756bab7b36989622ee777e9b60a26144325d54a53
Jul 01 14:40:43.223334 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:43.223297    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqm7n" event={"ID":"ee6ca283-3584-4392-af96-b83cf6a4b97a","Type":"ContainerStarted","Data":"25df08b693d436e6329132246f20c1b564d3329d821cb40f85cf4c3f5cee49d4"}
Jul 01 14:40:43.224607 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:43.224577    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"e29aed816cfd2b625cd6a1b756bab7b36989622ee777e9b60a26144325d54a53"}
Jul 01 14:40:43.226827 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:43.226800    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f" event={"ID":"383a8cfa-3e3e-4a5a-a5c8-91cca1468071","Type":"ContainerStarted","Data":"75711677663d09b00a41f471f6d7bb617c12274f6571fd74ef2e1a5727c32ecb"}
Jul 01 14:40:43.247532 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:43.247471    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-d5g9f" podStartSLOduration=1.305475688 podStartE2EDuration="2.247453939s" totalImagesPullingTime="941.978251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:43.246287984 +0000 UTC m=+91.962672742" watchObservedRunningTime="2026-07-01 14:40:43.247453939 +0000 UTC m=+91.963838695"
Jul 01 14:40:44.123331 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.123296    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:44.129823 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.129800    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:40:44.223358 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.223323    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:44.237100 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.237068    2541 generic.go:356] "Generic (PLEG): container finished" podID="ee6ca283-3584-4392-af96-b83cf6a4b97a" containerID="f81d9bca913d36a215b39037421dc22aaf1316b69762abca805ff55221b70f21" exitCode=0
Jul 01 14:40:44.237268 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.237122    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqm7n" event={"ID":"ee6ca283-3584-4392-af96-b83cf6a4b97a","Type":"ContainerDied","Data":"f81d9bca913d36a215b39037421dc22aaf1316b69762abca805ff55221b70f21"}
Jul 01 14:40:44.313887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.313836    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:44.321043 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.321020    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-9b98ff569-q7qhl"]
Jul 01 14:40:44.324727 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.324706    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.329042 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.329020    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 01 14:40:44.329139 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.329022    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 01 14:40:44.330098 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.330081    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 01 14:40:44.330215 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.330171    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 01 14:40:44.330497 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.330480    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ftebtrv8nhko\""
Jul 01 14:40:44.330615 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.330551    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-45xzn\""
Jul 01 14:40:44.330758 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.330747    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 01 14:40:44.339596 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339573    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339699 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339611    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339699 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339644    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339696    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339748    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbrph\" (UniqueName: \"kubernetes.io/projected/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-kube-api-access-gbrph\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339878 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339774    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-metrics-client-ca\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.339878 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.339863    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-grpc-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.340603 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.340564    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.344418 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.344398    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9b98ff569-q7qhl"]
Jul 01 14:40:44.441390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441340    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441397    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbrph\" (UniqueName: \"kubernetes.io/projected/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-kube-api-access-gbrph\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441427    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-metrics-client-ca\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441460    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-grpc-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441491    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441548    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441586    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.441630 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.441617    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.442509 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.442454    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-metrics-client-ca\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.445528 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.445022    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.445528 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.445170    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.445528 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.445470    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.445714 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.445590    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-grpc-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.445883 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.445818    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.446475 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.446448    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-secret-thanos-querier-tls\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.451960 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.451934    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbrph\" (UniqueName: \"kubernetes.io/projected/a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec-kube-api-access-gbrph\") pod \"thanos-querier-9b98ff569-q7qhl\" (UID: \"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec\") " pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.644941 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.644903    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:44.768193 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.768157    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:44.806426 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.806393    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9b98ff569-q7qhl"]
Jul 01 14:40:44.867458 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:44.867409    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:44.990203 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:44.990122    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c010ab_6ede_47ce_8e75_e2c1f2cc9aec.slice/crio-3ec0e843178f255a80bd57e014ba456f299d1327c35c9321c3cb0d7e3a5dccaf WatchSource:0}: Error finding container 3ec0e843178f255a80bd57e014ba456f299d1327c35c9321c3cb0d7e3a5dccaf: Status 404 returned error can't find the container with id 3ec0e843178f255a80bd57e014ba456f299d1327c35c9321c3cb0d7e3a5dccaf
Jul 01 14:40:45.242015 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.241928    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4" exitCode=0
Jul 01 14:40:45.242427 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.242016    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"}
Jul 01 14:40:45.243288 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.243267    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"3ec0e843178f255a80bd57e014ba456f299d1327c35c9321c3cb0d7e3a5dccaf"}
Jul 01 14:40:45.245494 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.245477    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqm7n" event={"ID":"ee6ca283-3584-4392-af96-b83cf6a4b97a","Type":"ContainerStarted","Data":"a850b2a15112a7464b59e343c70b4796efd8f9bc658324311c10446ef98ffe73"}
Jul 01 14:40:45.245596 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.245510    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqm7n" event={"ID":"ee6ca283-3584-4392-af96-b83cf6a4b97a","Type":"ContainerStarted","Data":"4572ddb93f989786dcba74c826db41efd2a1a05c8987d3b680e7b3db4a5f9521"}
Jul 01 14:40:45.294463 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.294409    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dqm7n" podStartSLOduration=3.153041445 podStartE2EDuration="4.294386532s" totalImagesPullingTime="1.068830697s" totalInitContainerRuntime="72.51439ms" isStatefulPod=true podCreationTimestamp="2026-07-01 14:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:45.293116329 +0000 UTC m=+94.009501086" watchObservedRunningTime="2026-07-01 14:40:45.294386532 +0000 UTC m=+94.010771291"
Jul 01 14:40:45.552180 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.552098    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:45.660811 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.660774    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:45.887125 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.887030    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86f44597f8-lw42k"]
Jul 01 14:40:45.889236 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.889217    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.893181 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893162    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 01 14:40:45.893867 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893817    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 01 14:40:45.893867 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893834    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 01 14:40:45.894010 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893819    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fhcqw\""
Jul 01 14:40:45.894010 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893828    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6lf4tldjdr6u2\""
Jul 01 14:40:45.894109 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.893821    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 01 14:40:45.913062 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.908716    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86f44597f8-lw42k"]
Jul 01 14:40:45.953817 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.953778    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.953901    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-client-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.953932    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxbq6\" (UniqueName: \"kubernetes.io/projected/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-kube-api-access-qxbq6\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954013 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.953959    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-client-certs\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954184 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.954040    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-audit-log\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954184 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.954114    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-tls\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:45.954184 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:45.954144    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-metrics-server-audit-profiles\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.054725 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054692    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-tls\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.054924 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054732    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-metrics-server-audit-profiles\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.054924 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054833    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.054924 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054917    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-client-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.055097 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054942    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxbq6\" (UniqueName: \"kubernetes.io/projected/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-kube-api-access-qxbq6\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.055097 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.054967    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-client-certs\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.055097 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.055000    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-audit-log\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.055795 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.055762    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.055960 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.055803    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-audit-log\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.056052 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.055991    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-metrics-server-audit-profiles\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.058028 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.057982    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-tls\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.058516 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.058491    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-client-ca-bundle\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.058600 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.058524    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-secret-metrics-server-client-certs\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.075499 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.075466    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxbq6\" (UniqueName: \"kubernetes.io/projected/2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e-kube-api-access-qxbq6\") pod \"metrics-server-86f44597f8-lw42k\" (UID: \"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e\") " pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.106059 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.106021    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:46.132035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.131979    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"]
Jul 01 14:40:46.135525 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.135501    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.141566 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.141272    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 01 14:40:46.141566 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.141338    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-47q9g\""
Jul 01 14:40:46.152740 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.152681    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"]
Jul 01 14:40:46.155879 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.155778    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc507241-b77e-4c75-9a91-af447ff5f2c0-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.155879 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.155854    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqt9f\" (UniqueName: \"kubernetes.io/projected/fc507241-b77e-4c75-9a91-af447ff5f2c0-kube-api-access-mqt9f\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.196676 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.196643    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:40:46.198705 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.198670    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:40:46.199234 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.199214    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.203129 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.202704    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 14:40:46.203129 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.203007    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 14:40:46.203544 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.203524    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 14:40:46.204012 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.203993    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 14:40:46.204094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.204063    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 14:40:46.204194 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.203999    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x7qb4\""
Jul 01 14:40:46.205328 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.205306    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 14:40:46.206069 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.205886    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 14:40:46.212764 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.212739    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 14:40:46.217902 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.217873    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:46.225251 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.225229    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256503    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256552    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc507241-b77e-4c75-9a91-af447ff5f2c0-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256583    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256604    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256619    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256643    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqt9f\" (UniqueName: \"kubernetes.io/projected/fc507241-b77e-4c75-9a91-af447ff5f2c0-kube-api-access-mqt9f\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256686    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck42r\" (UniqueName: \"kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256728    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.257301 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.256748    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.261624 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.261593    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc507241-b77e-4c75-9a91-af447ff5f2c0-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.274763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.274706    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqt9f\" (UniqueName: \"kubernetes.io/projected/fc507241-b77e-4c75-9a91-af447ff5f2c0-kube-api-access-mqt9f\") pod \"monitoring-plugin-cd8656b8-fbjnc\" (UID: \"fc507241-b77e-4c75-9a91-af447ff5f2c0\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.358053 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358015    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358227 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358066    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358227 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358199    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358321 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358231    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358321 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358278    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck42r\" (UniqueName: \"kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358380 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358335    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358380 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358372    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.358831 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.358804    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.359051 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.359031    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.359303 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.359091    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.359365 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.359331    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.361071 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.361053    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.361390 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.361373    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.372438 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.372397    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck42r\" (UniqueName: \"kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r\") pod \"console-6985db9469-gklmd\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") " pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.374235 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.374218    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86f44597f8-lw42k"]
Jul 01 14:40:46.448627 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.448598    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:46.523759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.523569    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:46.646811 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:46.646774    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:46.815885 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:46.815779    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a63bef6_9b82_4748_8bcc_c91a8fa0ec2e.slice/crio-a4745908049b3bd2351ac869d5dbbb9751dd438becafd50a0ce5f1bb96ae1eda WatchSource:0}: Error finding container a4745908049b3bd2351ac869d5dbbb9751dd438becafd50a0ce5f1bb96ae1eda: Status 404 returned error can't find the container with id a4745908049b3bd2351ac869d5dbbb9751dd438becafd50a0ce5f1bb96ae1eda
Jul 01 14:40:47.176090 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.176054    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:47.260744 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.260686    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k" event={"ID":"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e","Type":"ContainerStarted","Data":"a4745908049b3bd2351ac869d5dbbb9751dd438becafd50a0ce5f1bb96ae1eda"}
Jul 01 14:40:47.294176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.292144    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:47.367192 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.367132    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"]
Jul 01 14:40:47.371566 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:47.371538    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc507241_b77e_4c75_9a91_af447ff5f2c0.slice/crio-83cabd7da840226575210e8f1108961b5568d80f5e76a4b062327bc06779a357 WatchSource:0}: Error finding container 83cabd7da840226575210e8f1108961b5568d80f5e76a4b062327bc06779a357: Status 404 returned error can't find the container with id 83cabd7da840226575210e8f1108961b5568d80f5e76a4b062327bc06779a357
Jul 01 14:40:47.391702 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.391678    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:40:47.395193 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:40:47.395171    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96287955_c863_4afe_bc77_c2df7ba443b3.slice/crio-55a8670c9fa164cb05fe1648fc22cf5f172764afe4a2f71311ea74b47df9dfcf WatchSource:0}: Error finding container 55a8670c9fa164cb05fe1648fc22cf5f172764afe4a2f71311ea74b47df9dfcf: Status 404 returned error can't find the container with id 55a8670c9fa164cb05fe1648fc22cf5f172764afe4a2f71311ea74b47df9dfcf
Jul 01 14:40:47.679192 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:47.679043    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:48.268299 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.268261    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"9142f2c2234173b99ad70242e81825a7b9492ffc038ac3170c5de1f35e77ffae"}
Jul 01 14:40:48.268708 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.268330    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"775b610bed9300bd6ffc7e6f0b446af2e33e06a6e3c73b984748a6190d19401d"}
Jul 01 14:40:48.268708 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.268366    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"765ed6021795726928b71acd8c64e5116a9e2a7453ae400f209568c3efee5ec9"}
Jul 01 14:40:48.269569 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.269542    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6985db9469-gklmd" event={"ID":"96287955-c863-4afe-bc77-c2df7ba443b3","Type":"ContainerStarted","Data":"55a8670c9fa164cb05fe1648fc22cf5f172764afe4a2f71311ea74b47df9dfcf"}
Jul 01 14:40:48.270675 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.270650    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc" event={"ID":"fc507241-b77e-4c75-9a91-af447ff5f2c0","Type":"ContainerStarted","Data":"83cabd7da840226575210e8f1108961b5568d80f5e76a4b062327bc06779a357"}
Jul 01 14:40:48.273759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.273739    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"}
Jul 01 14:40:48.273868 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.273795    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"}
Jul 01 14:40:48.273868 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.273823    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"}
Jul 01 14:40:48.273976 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.273877    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"}
Jul 01 14:40:48.273976 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.273902    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"}
Jul 01 14:40:48.427749 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.427373    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:40:48.554519 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.554424    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:48.742985 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:48.742964    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:49.283218 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.283177    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"ffab3a48a1a6e35d23098d8e58e3d49b638d041eb184e9994827f80238b589bf"}
Jul 01 14:40:49.283657 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.283259    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"644b3d9578477e43d95f1ba05c94a8b03321d155653ee70f5a3ba2ccc41a96c6"}
Jul 01 14:40:49.283657 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.283298    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" event={"ID":"a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec","Type":"ContainerStarted","Data":"036d032248e4510cc93ee559904f02dff86cba297109c373907d3015da47511e"}
Jul 01 14:40:49.283657 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.283456    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:49.285540 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.285493    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k" event={"ID":"2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e","Type":"ContainerStarted","Data":"b6f13d87f13406451b7277ede0395ea05ed2f7349ef5f292eef8b20ab5f8e18f"}
Jul 01 14:40:49.290142 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.290074    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerStarted","Data":"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"}
Jul 01 14:40:49.308860 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.308792    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl" podStartSLOduration=1.839135008 podStartE2EDuration="5.308777072s" totalImagesPullingTime="3.469642064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:49.306662423 +0000 UTC m=+98.023047180" watchObservedRunningTime="2026-07-01 14:40:49.308777072 +0000 UTC m=+98.025161829"
Jul 01 14:40:49.342475 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.342412    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.184550457 podStartE2EDuration="7.342388925s" totalImagesPullingTime="5.143154674s" totalInitContainerRuntime="14.683794ms" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:49.339530325 +0000 UTC m=+98.055915081" watchObservedRunningTime="2026-07-01 14:40:49.342388925 +0000 UTC m=+98.058773684"
Jul 01 14:40:49.363451 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.363360    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k" podStartSLOduration=2.4699883480000002 podStartE2EDuration="4.363334597s" totalImagesPullingTime="1.893346249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:49.36242032 +0000 UTC m=+98.078805080" watchObservedRunningTime="2026-07-01 14:40:49.363334597 +0000 UTC m=+98.079719357"
Jul 01 14:40:49.475746 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.475721    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:49.584549 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:49.584475    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:50.068452 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.068422    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dz29r"
Jul 01 14:40:50.297431 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.297388    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc" event={"ID":"fc507241-b77e-4c75-9a91-af447ff5f2c0","Type":"ContainerStarted","Data":"b9fcdcbdce40dc9987a508a51c39073460c3e1540cc9688527b48695f30da452"}
Jul 01 14:40:50.297889 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.297457    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:50.302477 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.302453    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc"
Jul 01 14:40:50.315136 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.315092    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fbjnc" podStartSLOduration=2.268327744 podStartE2EDuration="4.315079205s" totalImagesPullingTime="2.046751461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:50.314630788 +0000 UTC m=+99.031015546" watchObservedRunningTime="2026-07-01 14:40:50.315079205 +0000 UTC m=+99.031463961"
Jul 01 14:40:50.534080 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:50.534043    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:51.297640 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:51.297591    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:53.250951 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.250915    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:53.312556 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.312517    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6985db9469-gklmd" event={"ID":"96287955-c863-4afe-bc77-c2df7ba443b3","Type":"ContainerStarted","Data":"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"}
Jul 01 14:40:53.333231 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.333176    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6985db9469-gklmd" podStartSLOduration=1.522479552 podStartE2EDuration="7.333159082s" totalImagesPullingTime="5.81067953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:53.33247668 +0000 UTC m=+102.048861687" watchObservedRunningTime="2026-07-01 14:40:53.333159082 +0000 UTC m=+102.049543839"
Jul 01 14:40:53.343574 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.343546    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:53.692079 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.692044    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:53.770687 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:53.770648    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:40:55.304017 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:55.303991    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-9b98ff569-q7qhl"
Jul 01 14:40:56.524775 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:56.524739    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:56.525196 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:56.524820    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:56.529509 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:56.529491    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:57.338759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:57.338729    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:40:57.575654 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:40:57.575627    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:00.232032 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:00.231999    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:00.294650 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:00.294624    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:00.540763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:00.540678    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:00.641282 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:00.641248    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:06.199791 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:06.199747    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:41:06.200212 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:06.199867    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:41:13.455447 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.455380    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" podUID="380ab168-967e-4928-b415-0467d4be0235" containerName="registry" containerID="cri-o://5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584" gracePeriod=30
Jul 01 14:41:13.707598 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.707543    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:41:13.804905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.804837    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.804918    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.804937    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzg67\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.804956    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.804983    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805094 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.805034    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805358 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.805099    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805358 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.805144    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca\") pod \"380ab168-967e-4928-b415-0467d4be0235\" (UID: \"380ab168-967e-4928-b415-0467d4be0235\") "
Jul 01 14:41:13.805507 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.805479    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:13.805746 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.805724    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:13.807676 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.807645    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:41:13.807791 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.807709    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67" (OuterVolumeSpecName: "kube-api-access-pzg67") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "kube-api-access-pzg67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:41:13.807891 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.807794    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:13.807891 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.807878    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:13.807984 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.807957    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:41:13.813799 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.813774    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "380ab168-967e-4928-b415-0467d4be0235" (UID: "380ab168-967e-4928-b415-0467d4be0235"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:41:13.906677 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906649    2541 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-installation-pull-secrets\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906677 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906676    2541 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-bound-sa-token\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906677 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906687    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzg67\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-kube-api-access-pzg67\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906709    2541 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/380ab168-967e-4928-b415-0467d4be0235-image-registry-private-configuration\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906719    2541 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-registry-certificates\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906727    2541 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/380ab168-967e-4928-b415-0467d4be0235-ca-trust-extracted\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906736    2541 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/380ab168-967e-4928-b415-0467d4be0235-registry-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.906917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:13.906744    2541 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/380ab168-967e-4928-b415-0467d4be0235-trusted-ca\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:41:14.404993 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.404964    2541 generic.go:356] "Generic (PLEG): container finished" podID="380ab168-967e-4928-b415-0467d4be0235" containerID="5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584" exitCode=0
Jul 01 14:41:14.405176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.405010    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp"
Jul 01 14:41:14.405176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.405047    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" event={"ID":"380ab168-967e-4928-b415-0467d4be0235","Type":"ContainerDied","Data":"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"}
Jul 01 14:41:14.405176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.405114    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd4cb4944-s82kp" event={"ID":"380ab168-967e-4928-b415-0467d4be0235","Type":"ContainerDied","Data":"451e915fbe9e6611aa1a395ee97c3d7f77577da09a7310fcef871867f42b656f"}
Jul 01 14:41:14.405176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.405143    2541 scope.go:117] "RemoveContainer" containerID="5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"
Jul 01 14:41:14.414159 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.414142    2541 scope.go:117] "RemoveContainer" containerID="5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"
Jul 01 14:41:14.414438 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:41:14.414421    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584\": container with ID starting with 5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584 not found: ID does not exist" containerID="5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"
Jul 01 14:41:14.414506 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.414447    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584"} err="failed to get container status \"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584\": rpc error: code = NotFound desc = could not find container \"5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584\": container with ID starting with 5c0593ae15296bf400516ae1e435b1ba1c7995909b4de748e0fde8097cb02584 not found: ID does not exist"
Jul 01 14:41:14.426575 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.426553    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:41:14.430671 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:14.430652    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-cd4cb4944-s82kp"]
Jul 01 14:41:15.548087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:15.548053    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:15.612653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:15.612626    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:41:15.724200 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:15.724169    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="380ab168-967e-4928-b415-0467d4be0235" path="/var/lib/kubelet/pods/380ab168-967e-4928-b415-0467d4be0235/volumes"
Jul 01 14:41:18.424469 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:18.424385    2541 generic.go:356] "Generic (PLEG): container finished" podID="b2b0c7e8-e90e-4093-ae0f-82e75413c8cf" containerID="4b1c050dcba855506234a47d5598695cf708014a8f858be148c68c3ef209a9de" exitCode=0
Jul 01 14:41:18.424469 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:18.424447    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-7qj2r" event={"ID":"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf","Type":"ContainerDied","Data":"4b1c050dcba855506234a47d5598695cf708014a8f858be148c68c3ef209a9de"}
Jul 01 14:41:18.424924 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:18.424817    2541 scope.go:117] "RemoveContainer" containerID="4b1c050dcba855506234a47d5598695cf708014a8f858be148c68c3ef209a9de"
Jul 01 14:41:19.430610 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:19.430581    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-7qj2r" event={"ID":"b2b0c7e8-e90e-4093-ae0f-82e75413c8cf","Type":"ContainerStarted","Data":"d7aaf6d3e062a9d95150ec6cd26c550ef97bc44ca93705ee810ca9b3c008f129"}
Jul 01 14:41:23.448150 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:23.448101    2541 generic.go:356] "Generic (PLEG): container finished" podID="4a922e63-f7a9-4190-9acc-20b9c68974cd" containerID="3227d5e0c178e6364559827d4d938f057d2444d57990e8993196f51f160087ed" exitCode=0
Jul 01 14:41:23.448624 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:23.448182    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s" event={"ID":"4a922e63-f7a9-4190-9acc-20b9c68974cd","Type":"ContainerDied","Data":"3227d5e0c178e6364559827d4d938f057d2444d57990e8993196f51f160087ed"}
Jul 01 14:41:23.448624 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:23.448586    2541 scope.go:117] "RemoveContainer" containerID="3227d5e0c178e6364559827d4d938f057d2444d57990e8993196f51f160087ed"
Jul 01 14:41:24.454890 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:24.454827    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-7d96s" event={"ID":"4a922e63-f7a9-4190-9acc-20b9c68974cd","Type":"ContainerStarted","Data":"c03bfebb8210390d59f3f3e442265206856fa2aeed583136d349563db8a3c400"}
Jul 01 14:41:26.204317 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:26.204286    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:41:26.208253 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:26.208229    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86f44597f8-lw42k"
Jul 01 14:41:33.497687 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:33.497648    2541 generic.go:356] "Generic (PLEG): container finished" podID="61c6cdfc-f872-43a1-ace1-ee78a7c8b5af" containerID="a6260cb76284615df6dbcbda57447a6af7952357b70daa62916d5c3176656216" exitCode=0
Jul 01 14:41:33.498092 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:33.497719    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x" event={"ID":"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af","Type":"ContainerDied","Data":"a6260cb76284615df6dbcbda57447a6af7952357b70daa62916d5c3176656216"}
Jul 01 14:41:33.498142 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:33.498091    2541 scope.go:117] "RemoveContainer" containerID="a6260cb76284615df6dbcbda57447a6af7952357b70daa62916d5c3176656216"
Jul 01 14:41:34.503880 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:41:34.503823    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-r8q4x" event={"ID":"61c6cdfc-f872-43a1-ace1-ee78a7c8b5af","Type":"ContainerStarted","Data":"b9df3d6aa2a76b8171981e27f5f7a32a8e68844f2c6924cd3dc0b8465ffb1c74"}
Jul 01 14:42:00.618294 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:00.618258    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:00.740267 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:00.740238    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:01.746557 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746525    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:01.746987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746821    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="alertmanager" containerID="cri-o://d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680" gracePeriod=120
Jul 01 14:42:01.746987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746868    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-web" containerID="cri-o://6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3" gracePeriod=120
Jul 01 14:42:01.746987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746889    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="prom-label-proxy" containerID="cri-o://199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107" gracePeriod=120
Jul 01 14:42:01.746987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746908    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="config-reloader" containerID="cri-o://1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee" gracePeriod=120
Jul 01 14:42:01.747199 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746976    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-metric" containerID="cri-o://e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a" gracePeriod=120
Jul 01 14:42:01.747199 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:01.746972    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy" containerID="cri-o://2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46" gracePeriod=120
Jul 01 14:42:02.623653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623619    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107" exitCode=0
Jul 01 14:42:02.623653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623648    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46" exitCode=0
Jul 01 14:42:02.623653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623656    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee" exitCode=0
Jul 01 14:42:02.623653 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623662    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680" exitCode=0
Jul 01 14:42:02.623961 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623685    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"}
Jul 01 14:42:02.623961 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623751    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"}
Jul 01 14:42:02.623961 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623777    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"}
Jul 01 14:42:02.623961 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:02.623801    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"}
Jul 01 14:42:03.021025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.021000    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.137504 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137467    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137504 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137508    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137530    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137573    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137598    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137627    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137651    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137690    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.137751 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137732    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.138234 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137765    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.138234 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137807    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f6lf\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.138234 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137870    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.138234 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.137904    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db\") pod \"9f501d2f-2806-4eac-8dbb-423a6759d66d\" (UID: \"9f501d2f-2806-4eac-8dbb-423a6759d66d\") "
Jul 01 14:42:03.138700 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.138653    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:42:03.139001 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.138490    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:03.139093 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.139014    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:03.141008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.140976    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.141254 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.141223    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.141364 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.141341    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf" (OuterVolumeSpecName: "kube-api-access-4f6lf") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "kube-api-access-4f6lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:42:03.141582 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.141555    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:42:03.141671 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.141649    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.141931 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.141911    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume" (OuterVolumeSpecName: "config-volume") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.142562 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.142541    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.142612 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.142541    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out" (OuterVolumeSpecName: "config-out") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:42:03.146113 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.146092    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.152585 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.152564    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config" (OuterVolumeSpecName: "web-config") pod "9f501d2f-2806-4eac-8dbb-423a6759d66d" (UID: "9f501d2f-2806-4eac-8dbb-423a6759d66d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238801    2541 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-main-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238873    2541 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-cluster-tls-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238884    2541 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238895    2541 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-web-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238903    2541 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-out\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238910    2541 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-config-volume\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.238905 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238917    2541 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-tls-assets\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238925    2541 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238934    2541 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238944    2541 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f501d2f-2806-4eac-8dbb-423a6759d66d-metrics-client-ca\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238952    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f6lf\" (UniqueName: \"kubernetes.io/projected/9f501d2f-2806-4eac-8dbb-423a6759d66d-kube-api-access-4f6lf\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238960    2541 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f501d2f-2806-4eac-8dbb-423a6759d66d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.239241 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.238968    2541 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f501d2f-2806-4eac-8dbb-423a6759d66d-alertmanager-main-db\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:03.631370 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631278    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a" exitCode=0
Jul 01 14:42:03.631370 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631319    2541 generic.go:356] "Generic (PLEG): container finished" podID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerID="6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3" exitCode=0
Jul 01 14:42:03.631370 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631362    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.631669 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631356    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"}
Jul 01 14:42:03.631669 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631504    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"}
Jul 01 14:42:03.631669 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631551    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f501d2f-2806-4eac-8dbb-423a6759d66d","Type":"ContainerDied","Data":"e29aed816cfd2b625cd6a1b756bab7b36989622ee777e9b60a26144325d54a53"}
Jul 01 14:42:03.631669 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.631572    2541 scope.go:117] "RemoveContainer" containerID="199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"
Jul 01 14:42:03.640487 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.640464    2541 scope.go:117] "RemoveContainer" containerID="e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"
Jul 01 14:42:03.648766 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.648747    2541 scope.go:117] "RemoveContainer" containerID="2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"
Jul 01 14:42:03.656131 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.656112    2541 scope.go:117] "RemoveContainer" containerID="6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"
Jul 01 14:42:03.663288 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.663268    2541 scope.go:117] "RemoveContainer" containerID="1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"
Jul 01 14:42:03.670448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.670424    2541 scope.go:117] "RemoveContainer" containerID="d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"
Jul 01 14:42:03.677820 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.677795    2541 scope.go:117] "RemoveContainer" containerID="c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"
Jul 01 14:42:03.686244 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.686202    2541 scope.go:117] "RemoveContainer" containerID="199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"
Jul 01 14:42:03.686611 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.686585    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107\": container with ID starting with 199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107 not found: ID does not exist" containerID="199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"
Jul 01 14:42:03.686721 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.686625    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"} err="failed to get container status \"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107\": rpc error: code = NotFound desc = could not find container \"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107\": container with ID starting with 199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107 not found: ID does not exist"
Jul 01 14:42:03.686721 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.686651    2541 scope.go:117] "RemoveContainer" containerID="e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"
Jul 01 14:42:03.687058 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.687030    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a\": container with ID starting with e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a not found: ID does not exist" containerID="e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"
Jul 01 14:42:03.687153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687067    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"} err="failed to get container status \"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a\": rpc error: code = NotFound desc = could not find container \"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a\": container with ID starting with e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a not found: ID does not exist"
Jul 01 14:42:03.687153 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687090    2541 scope.go:117] "RemoveContainer" containerID="2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"
Jul 01 14:42:03.687395 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.687377    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46\": container with ID starting with 2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46 not found: ID does not exist" containerID="2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"
Jul 01 14:42:03.687434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687400    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"} err="failed to get container status \"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46\": rpc error: code = NotFound desc = could not find container \"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46\": container with ID starting with 2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46 not found: ID does not exist"
Jul 01 14:42:03.687434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687416    2541 scope.go:117] "RemoveContainer" containerID="6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"
Jul 01 14:42:03.687693 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.687677    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3\": container with ID starting with 6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3 not found: ID does not exist" containerID="6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"
Jul 01 14:42:03.687736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687699    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"} err="failed to get container status \"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3\": rpc error: code = NotFound desc = could not find container \"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3\": container with ID starting with 6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3 not found: ID does not exist"
Jul 01 14:42:03.687736 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687715    2541 scope.go:117] "RemoveContainer" containerID="1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"
Jul 01 14:42:03.687956 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.687941    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee\": container with ID starting with 1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee not found: ID does not exist" containerID="1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"
Jul 01 14:42:03.688011 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687960    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"} err="failed to get container status \"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee\": rpc error: code = NotFound desc = could not find container \"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee\": container with ID starting with 1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee not found: ID does not exist"
Jul 01 14:42:03.688011 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.687974    2541 scope.go:117] "RemoveContainer" containerID="d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"
Jul 01 14:42:03.688166 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.688152    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680\": container with ID starting with d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680 not found: ID does not exist" containerID="d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"
Jul 01 14:42:03.688247 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688167    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"} err="failed to get container status \"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680\": rpc error: code = NotFound desc = could not find container \"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680\": container with ID starting with d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680 not found: ID does not exist"
Jul 01 14:42:03.688247 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688179    2541 scope.go:117] "RemoveContainer" containerID="c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"
Jul 01 14:42:03.688362 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:03.688349    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4\": container with ID starting with c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4 not found: ID does not exist" containerID="c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"
Jul 01 14:42:03.688401 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688363    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"} err="failed to get container status \"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4\": rpc error: code = NotFound desc = could not find container \"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4\": container with ID starting with c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4 not found: ID does not exist"
Jul 01 14:42:03.688401 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688374    2541 scope.go:117] "RemoveContainer" containerID="199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"
Jul 01 14:42:03.688600 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688581    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107"} err="failed to get container status \"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107\": rpc error: code = NotFound desc = could not find container \"199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107\": container with ID starting with 199c383ff3e1811f504a5bde3e3b9de70a360053d7720b1e543d8025e0e7e107 not found: ID does not exist"
Jul 01 14:42:03.688656 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688601    2541 scope.go:117] "RemoveContainer" containerID="e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"
Jul 01 14:42:03.688818 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688803    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a"} err="failed to get container status \"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a\": rpc error: code = NotFound desc = could not find container \"e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a\": container with ID starting with e869a85c59e90054345afd2cb67fc347df5cb643e41adec698707325606bb08a not found: ID does not exist"
Jul 01 14:42:03.688887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.688818    2541 scope.go:117] "RemoveContainer" containerID="2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"
Jul 01 14:42:03.689030 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689014    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46"} err="failed to get container status \"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46\": rpc error: code = NotFound desc = could not find container \"2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46\": container with ID starting with 2faca9c71a5bde772260f5b3c83346dd79851ef5af284084d3f0e46458024e46 not found: ID does not exist"
Jul 01 14:42:03.689070 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689031    2541 scope.go:117] "RemoveContainer" containerID="6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"
Jul 01 14:42:03.689211 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689197    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3"} err="failed to get container status \"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3\": rpc error: code = NotFound desc = could not find container \"6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3\": container with ID starting with 6ac1e85010b922f68af4de8c66ff486b7942a3982070aee3c9060ad3abde02f3 not found: ID does not exist"
Jul 01 14:42:03.689257 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689212    2541 scope.go:117] "RemoveContainer" containerID="1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"
Jul 01 14:42:03.689383 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689368    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee"} err="failed to get container status \"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee\": rpc error: code = NotFound desc = could not find container \"1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee\": container with ID starting with 1b04025d5731ac259ca76b04c1b869f124caa6aaa68358d4d4aa8334f47456ee not found: ID does not exist"
Jul 01 14:42:03.689425 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689383    2541 scope.go:117] "RemoveContainer" containerID="d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"
Jul 01 14:42:03.689590 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689571    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680"} err="failed to get container status \"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680\": rpc error: code = NotFound desc = could not find container \"d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680\": container with ID starting with d0cefb8c3c3565fecd083564a9e332aa3dcbcb947d6954619eab883701b1b680 not found: ID does not exist"
Jul 01 14:42:03.689662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689591    2541 scope.go:117] "RemoveContainer" containerID="c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"
Jul 01 14:42:03.689889 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.689868    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4"} err="failed to get container status \"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4\": rpc error: code = NotFound desc = could not find container \"c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4\": container with ID starting with c118a9b887191f9b0a739284727bd3217dcfa19bc36c6e169984160c248196e4 not found: ID does not exist"
Jul 01 14:42:03.695623 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.695600    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:03.697294 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.697254    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:03.724641 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.724610    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" path="/var/lib/kubelet/pods/9f501d2f-2806-4eac-8dbb-423a6759d66d/volumes"
Jul 01 14:42:03.831470 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831440    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:03.831815 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831798    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="init-config-reloader"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831817    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="init-config-reloader"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831829    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-web"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831855    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-web"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831875    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831884    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831895    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="prom-label-proxy"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831904    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="prom-label-proxy"
Jul 01 14:42:03.831918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831915    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-metric"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831924    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-metric"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831934    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="alertmanager"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831942    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="alertmanager"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831961    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380ab168-967e-4928-b415-0467d4be0235" containerName="registry"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831969    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="380ab168-967e-4928-b415-0467d4be0235" containerName="registry"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831983    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="config-reloader"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.831991    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="config-reloader"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832073    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-web"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832084    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832097    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="380ab168-967e-4928-b415-0467d4be0235" containerName="registry"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832106    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="config-reloader"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832114    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="alertmanager"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832124    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="kube-rbac-proxy-metric"
Jul 01 14:42:03.832305 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.832133    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f501d2f-2806-4eac-8dbb-423a6759d66d" containerName="prom-label-proxy"
Jul 01 14:42:03.839740 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.839711    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.845279 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845254    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 14:42:03.845487 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845441    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 14:42:03.845644 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845625    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 14:42:03.845726 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845694    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 14:42:03.845726 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845706    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 14:42:03.845885 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845693    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-thc26\""
Jul 01 14:42:03.845885 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845773    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 14:42:03.845885 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.845782    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 14:42:03.846111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.846052    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 14:42:03.852158 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.852140    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 14:42:03.859972 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.859952    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:03.944195 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944162    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944354 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944209    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-web-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944354 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944228    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-volume\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944354 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944274    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944354 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944330    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944486 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944369    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944486 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944386    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rplk9\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-kube-api-access-rplk9\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944486 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944407    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944486 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944432    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944602 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944500    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-out\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944602 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944517    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944602 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944540    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:03.944602 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:03.944567    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045166 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045129    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045166 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045171    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rplk9\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-kube-api-access-rplk9\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045195    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045231    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045284    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-out\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045309    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045332    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045358    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045384    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045434    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-web-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045478    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-volume\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045499    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.045693 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045523    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.046448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.045823    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.046448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.046323    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.046448 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.046419    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1157b362-a71a-4faa-9201-e0fa88fa5c05-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.048556    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-out\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.048691    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.048884    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.048921    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-config-volume\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.049042    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-web-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049417 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.049271    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049885 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.049835    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.049994 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.049974    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.050538 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.050522    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1157b362-a71a-4faa-9201-e0fa88fa5c05-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.058176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.058153    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rplk9\" (UniqueName: \"kubernetes.io/projected/1157b362-a71a-4faa-9201-e0fa88fa5c05-kube-api-access-rplk9\") pod \"alertmanager-main-0\" (UID: \"1157b362-a71a-4faa-9201-e0fa88fa5c05\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.149703 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.149663    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 14:42:04.305798 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.305776    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 14:42:04.307986 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:42:04.307959    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1157b362_a71a_4faa_9201_e0fa88fa5c05.slice/crio-05ae873fc6084777fa16128a1650fc75e69568151472fa5d7696b59b82cd32c3 WatchSource:0}: Error finding container 05ae873fc6084777fa16128a1650fc75e69568151472fa5d7696b59b82cd32c3: Status 404 returned error can't find the container with id 05ae873fc6084777fa16128a1650fc75e69568151472fa5d7696b59b82cd32c3
Jul 01 14:42:04.638734 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.638640    2541 generic.go:356] "Generic (PLEG): container finished" podID="1157b362-a71a-4faa-9201-e0fa88fa5c05" containerID="e704480ba86d717f291e1300d554e0bc7ff6b388ec64f379b346e2a0f7aa4011" exitCode=0
Jul 01 14:42:04.638923 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.638733    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerDied","Data":"e704480ba86d717f291e1300d554e0bc7ff6b388ec64f379b346e2a0f7aa4011"}
Jul 01 14:42:04.638923 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:04.638806    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"05ae873fc6084777fa16128a1650fc75e69568151472fa5d7696b59b82cd32c3"}
Jul 01 14:42:05.649343 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649304    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"1e1b5d84d26f89d1858caf26713e925a7efd8a10f60da60f3abc61c118a99e3f"}
Jul 01 14:42:05.649773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649386    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"2dad83aa16fffc755c80877b98fd967ebc47b3d747e29be721ac9f0625e8768d"}
Jul 01 14:42:05.649773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649430    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"442f431c2277b26f8565d57f2444750aa6e4ce5aefdb8c62ba1d63f61a2827c1"}
Jul 01 14:42:05.649773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649480    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"6f9ad6cfd97d850a11e24926db57020d2a49151504fabe325c78652f78d2585e"}
Jul 01 14:42:05.649773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649519    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"23ffe7154b89cc84bd5276205a54f25911d479a952998ebb37e2a6e838f21e7a"}
Jul 01 14:42:05.649773 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.649554    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1157b362-a71a-4faa-9201-e0fa88fa5c05","Type":"ContainerStarted","Data":"dac629ad58bba5d8f447d93a47c91a99b3547a901baeb2032ff1666f64e70199"}
Jul 01 14:42:05.777748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.777696    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.763012524 podStartE2EDuration="2.777679771s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.667247ms" isStatefulPod=false podCreationTimestamp="2026-07-01 14:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:42:05.705185283 +0000 UTC m=+174.421570040" watchObservedRunningTime="2026-07-01 14:42:05.777679771 +0000 UTC m=+174.494064576"
Jul 01 14:42:05.778925 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.778909    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cb57c58c-5twzt"]
Jul 01 14:42:05.782282 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.782264    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.789465 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.789429    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-j65h9\""
Jul 01 14:42:05.789644 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.789512    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 01 14:42:05.789644 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.789538    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 01 14:42:05.789829 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.789802    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 01 14:42:05.789950 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.789877    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 01 14:42:05.790446 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.790413    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 01 14:42:05.796829 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.796812    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 01 14:42:05.805414 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.805390    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cb57c58c-5twzt"]
Jul 01 14:42:05.861499 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861467    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-metrics-client-ca\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861513    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mffzt\" (UniqueName: \"kubernetes.io/projected/d913a3b9-c2eb-4e2c-b690-a202daef584f-kube-api-access-mffzt\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861545    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861578    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-federate-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861658 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861615    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861911 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861674    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861911 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861730    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.861911 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.861765    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-serving-certs-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963206 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963178    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963222    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-serving-certs-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963264    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-metrics-client-ca\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963292    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mffzt\" (UniqueName: \"kubernetes.io/projected/d913a3b9-c2eb-4e2c-b690-a202daef584f-kube-api-access-mffzt\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963319    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963350    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-federate-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963393    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.963457 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.963447    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.964182 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.964152    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-metrics-client-ca\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.964388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.964237    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.964388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.964282    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d913a3b9-c2eb-4e2c-b690-a202daef584f-serving-certs-ca-bundle\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.966012 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.965993    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-federate-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.966146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.966127    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-telemeter-client-tls\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.966364 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.966345    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.966364 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.966358    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d913a3b9-c2eb-4e2c-b690-a202daef584f-secret-telemeter-client\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:05.975754 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:05.975733    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mffzt\" (UniqueName: \"kubernetes.io/projected/d913a3b9-c2eb-4e2c-b690-a202daef584f-kube-api-access-mffzt\") pod \"telemeter-client-6cb57c58c-5twzt\" (UID: \"d913a3b9-c2eb-4e2c-b690-a202daef584f\") " pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:06.093008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:06.092978    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt"
Jul 01 14:42:06.233203 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:06.233182    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cb57c58c-5twzt"]
Jul 01 14:42:06.234559 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:42:06.234532    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd913a3b9_c2eb_4e2c_b690_a202daef584f.slice/crio-471e721a3b3f9a51a98fc71ac18320a0643a11525cd17802a14cc45901eefff2 WatchSource:0}: Error finding container 471e721a3b3f9a51a98fc71ac18320a0643a11525cd17802a14cc45901eefff2: Status 404 returned error can't find the container with id 471e721a3b3f9a51a98fc71ac18320a0643a11525cd17802a14cc45901eefff2
Jul 01 14:42:06.657129 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:06.657041    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt" event={"ID":"d913a3b9-c2eb-4e2c-b690-a202daef584f","Type":"ContainerStarted","Data":"471e721a3b3f9a51a98fc71ac18320a0643a11525cd17802a14cc45901eefff2"}
Jul 01 14:42:08.669770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:08.669717    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt" event={"ID":"d913a3b9-c2eb-4e2c-b690-a202daef584f","Type":"ContainerStarted","Data":"5b5bbe5329f34f4d39d1d895933132ee2e66d178817c11806c6c086f299b2498"}
Jul 01 14:42:08.670246 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:08.669801    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt" event={"ID":"d913a3b9-c2eb-4e2c-b690-a202daef584f","Type":"ContainerStarted","Data":"93ce3485f03adb8f3aa18ceb440d2be8be526140af93a7ec88a1a643fcd9663c"}
Jul 01 14:42:08.670246 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:08.669823    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt" event={"ID":"d913a3b9-c2eb-4e2c-b690-a202daef584f","Type":"ContainerStarted","Data":"3a499fad6f2075225f18a3c3228b5f873092b7eaf048e8e274cb9b77d5e3301c"}
Jul 01 14:42:08.697619 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:08.697233    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cb57c58c-5twzt" podStartSLOduration=1.843550881 podStartE2EDuration="3.697212836s" totalImagesPullingTime="1.853661955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:42:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:42:08.695523714 +0000 UTC m=+177.411908470" watchObservedRunningTime="2026-07-01 14:42:08.697212836 +0000 UTC m=+177.413597592"
Jul 01 14:42:09.587546 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:09.587514    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:09.815883 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:09.815852    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:19.955765 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:19.955724    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:42:20.105786 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:20.105755    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:20.208105 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:20.208017    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:20.308852 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:20.308806    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:34.122918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:34.122885    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:44.980371 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:44.980304    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6985db9469-gklmd" podUID="96287955-c863-4afe-bc77-c2df7ba443b3" containerName="console" containerID="cri-o://9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25" gracePeriod=15
Jul 01 14:42:45.224443 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.224413    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:42:45.300987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.300896    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.300987 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.300971    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301201 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.300997    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301201 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301025    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301201 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301099    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck42r\" (UniqueName: \"kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301201 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301142    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301201 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301164    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config\") pod \"96287955-c863-4afe-bc77-c2df7ba443b3\" (UID: \"96287955-c863-4afe-bc77-c2df7ba443b3\") "
Jul 01 14:42:45.301565 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301433    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:45.301565 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301432    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:45.301891 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301833    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:45.301981 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.301776    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config" (OuterVolumeSpecName: "console-config") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:42:45.303606 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.303572    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:45.303714 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.303686    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r" (OuterVolumeSpecName: "kube-api-access-ck42r") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "kube-api-access-ck42r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:42:45.303714 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.303704    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "96287955-c863-4afe-bc77-c2df7ba443b3" (UID: "96287955-c863-4afe-bc77-c2df7ba443b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:42:45.402763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402736    2541 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-oauth-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.402763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402760    2541 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-oauth-serving-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.402763 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402771    2541 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-service-ca\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.403019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402779    2541 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96287955-c863-4afe-bc77-c2df7ba443b3-console-serving-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.403019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402788    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck42r\" (UniqueName: \"kubernetes.io/projected/96287955-c863-4afe-bc77-c2df7ba443b3-kube-api-access-ck42r\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.403019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402796    2541 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-trusted-ca-bundle\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.403019 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.402805    2541 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96287955-c863-4afe-bc77-c2df7ba443b3-console-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:42:45.826145 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.826109    2541 generic.go:356] "Generic (PLEG): container finished" podID="96287955-c863-4afe-bc77-c2df7ba443b3" containerID="9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25" exitCode=0
Jul 01 14:42:45.826366 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.826166    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6985db9469-gklmd"
Jul 01 14:42:45.826366 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.826174    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6985db9469-gklmd" event={"ID":"96287955-c863-4afe-bc77-c2df7ba443b3","Type":"ContainerDied","Data":"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"}
Jul 01 14:42:45.826366 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.826258    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6985db9469-gklmd" event={"ID":"96287955-c863-4afe-bc77-c2df7ba443b3","Type":"ContainerDied","Data":"55a8670c9fa164cb05fe1648fc22cf5f172764afe4a2f71311ea74b47df9dfcf"}
Jul 01 14:42:45.826366 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.826328    2541 scope.go:117] "RemoveContainer" containerID="9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"
Jul 01 14:42:45.834857 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.834828    2541 scope.go:117] "RemoveContainer" containerID="9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"
Jul 01 14:42:45.835160 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:42:45.835140    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25\": container with ID starting with 9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25 not found: ID does not exist" containerID="9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"
Jul 01 14:42:45.835227 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.835169    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25"} err="failed to get container status \"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25\": rpc error: code = NotFound desc = could not find container \"9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25\": container with ID starting with 9299209de6503d3658a9336da28ad57fb0065ad5fbc0aa4e858273228c66da25 not found: ID does not exist"
Jul 01 14:42:45.845031 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.845010    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:42:45.848972 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:45.848953    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6985db9469-gklmd"]
Jul 01 14:42:47.724566 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:47.724533    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96287955-c863-4afe-bc77-c2df7ba443b3" path="/var/lib/kubelet/pods/96287955-c863-4afe-bc77-c2df7ba443b3/volumes"
Jul 01 14:42:49.488487 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:49.488457    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:42:49.576190 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:42:49.576156    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:00.780641 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:00.780610    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:15.961599 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:15.961565    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:16.037677 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:16.037645    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:22.855038 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.854998    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:43:22.855612 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.855592    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96287955-c863-4afe-bc77-c2df7ba443b3" containerName="console"
Jul 01 14:43:22.855670 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.855616    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="96287955-c863-4afe-bc77-c2df7ba443b3" containerName="console"
Jul 01 14:43:22.855817 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.855798    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="96287955-c863-4afe-bc77-c2df7ba443b3" containerName="console"
Jul 01 14:43:22.860328 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.860289    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.863344 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.863315    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 14:43:22.864515 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864491    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 14:43:22.864633 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864510    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 14:43:22.864633 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864538    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 14:43:22.864633 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864580    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 14:43:22.864633 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864603    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 14:43:22.864633 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864540    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 14:43:22.864930 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.864912    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x7qb4\""
Jul 01 14:43:22.874757 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.874734    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:43:22.883425 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.883402    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 14:43:22.916127 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916092    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916139    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916230    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916283    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk79w\" (UniqueName: \"kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916454 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916329    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916454 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916375    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.916454 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.916395    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:22.942540 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:22.942503    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:23.008631 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.008599    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:23.017784 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017760    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.017893 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017793    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.017893 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017812    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.017893 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017871    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018003 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017916    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018003 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017962    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018003 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.017992    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk79w\" (UniqueName: \"kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.018597    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.018610    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.018620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.018623    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.019277 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.019254    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.020520 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.020499    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.020684 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.020667    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.027272 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.027250    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk79w\" (UniqueName: \"kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w\") pod \"console-67c54b9b7c-vtlsd\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") " pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.171929 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.171897    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:23.306830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.306803    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:43:23.309051 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:43:23.309010    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdc8cc3_2882_4471_8ef9_25f9db7c5c0e.slice/crio-a06bd95ae3840f33cdf7b8f841a87cff779e860cdde61243eb50a6f5f15efb94 WatchSource:0}: Error finding container a06bd95ae3840f33cdf7b8f841a87cff779e860cdde61243eb50a6f5f15efb94: Status 404 returned error can't find the container with id a06bd95ae3840f33cdf7b8f841a87cff779e860cdde61243eb50a6f5f15efb94
Jul 01 14:43:23.991136 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.991101    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c54b9b7c-vtlsd" event={"ID":"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e","Type":"ContainerStarted","Data":"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"}
Jul 01 14:43:23.991512 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:23.991152    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c54b9b7c-vtlsd" event={"ID":"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e","Type":"ContainerStarted","Data":"a06bd95ae3840f33cdf7b8f841a87cff779e860cdde61243eb50a6f5f15efb94"}
Jul 01 14:43:24.017372 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:24.017324    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67c54b9b7c-vtlsd" podStartSLOduration=2.017309292 podStartE2EDuration="2.017309292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:43:24.015767253 +0000 UTC m=+252.732152010" watchObservedRunningTime="2026-07-01 14:43:24.017309292 +0000 UTC m=+252.733694049"
Jul 01 14:43:27.992152 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:27.992121    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:28.062480 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:28.062445    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:30.550600 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:30.550567    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:30.742140 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:30.742110    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:33.172932 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:33.172859    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:33.172932 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:33.172917    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:33.177691 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:33.177670    2541 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:34.036052 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:34.036025    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:43:34.343463 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:34.343384    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:34.443992 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:34.443957    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:46.683428 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:46.683393    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:43:46.765972 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:43:46.765941    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:11.666426 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:11.666400    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:44:11.676274 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:11.676244    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:44:11.678982 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:11.678958    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:44:11.682086 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:11.682062    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:44:11.686028 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:11.686000    2541 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 14:44:24.375251 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:24.375208    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:25.728110 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:25.728081    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:40.992283 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:40.992247    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:41.094018 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:41.093989    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:52.273444 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:52.273400    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:52.789320 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:52.789287    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:57.535259 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:57.535208    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:57.815791 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:57.815711    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:57.989604 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:57.989568    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.094468 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.094385    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.194030 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.194000    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.655159 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.655127    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.664593 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.664570    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xskdt"]
Jul 01 14:44:58.667962 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.667947    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.670378 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.670353    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 01 14:44:58.670505 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.670445    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 01 14:44:58.670505 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.670445    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 01 14:44:58.671491 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.671471    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 01 14:44:58.671605 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.671511    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-m5k4n\""
Jul 01 14:44:58.676634 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.676614    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xskdt"]
Jul 01 14:44:58.751504 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.751466    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-certificates\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.751711 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.751689    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jp4f\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-kube-api-access-2jp4f\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.755459 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.755428    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.852920 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.852886    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-certificates\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.853061 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.852969    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jp4f\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-kube-api-access-2jp4f\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.855252 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.855228    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:58.855760 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.855741    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-certificates\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.863258 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.863234    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jp4f\" (UniqueName: \"kubernetes.io/projected/e0d37c00-b9fe-44f3-b5fa-ef36717dbc93-kube-api-access-2jp4f\") pod \"keda-admission-5fbd5c4644-xskdt\" (UID: \"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93\") " pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:58.979214 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:58.979181    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:44:59.056732 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.056697    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:59.121309 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.121286    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xskdt"]
Jul 01 14:44:59.123593 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:44:59.123565    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d37c00_b9fe_44f3_b5fa_ef36717dbc93.slice/crio-b9d8b3c028b4be23bf9e63e6542e3b150f16525e23a27510c1ad249f4be2d7e6 WatchSource:0}: Error finding container b9d8b3c028b4be23bf9e63e6542e3b150f16525e23a27510c1ad249f4be2d7e6: Status 404 returned error can't find the container with id b9d8b3c028b4be23bf9e63e6542e3b150f16525e23a27510c1ad249f4be2d7e6
Jul 01 14:44:59.124860 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.124825    2541 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 14:44:59.382945 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.382857    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt" event={"ID":"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93","Type":"ContainerStarted","Data":"b9d8b3c028b4be23bf9e63e6542e3b150f16525e23a27510c1ad249f4be2d7e6"}
Jul 01 14:44:59.497857 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.497821    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:44:59.685776 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:44:59.685744    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:45:02.400433 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:45:02.400342    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt" event={"ID":"e0d37c00-b9fe-44f3-b5fa-ef36717dbc93","Type":"ContainerStarted","Data":"a0b591e87d7d78d10128e9493aaf0c2079efb79d39b3a396b907b8cd56a6d324"}
Jul 01 14:45:02.400790 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:45:02.400434    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:45:02.421045 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:45:02.420999    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt" podStartSLOduration=1.52470517 podStartE2EDuration="4.420982281s" totalImagesPullingTime="2.896277111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:45:02.419130264 +0000 UTC m=+351.135515021" watchObservedRunningTime="2026-07-01 14:45:02.420982281 +0000 UTC m=+351.137367037"
Jul 01 14:45:23.408133 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:45:23.408103    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-xskdt"
Jul 01 14:46:06.873910 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:06.873874    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:06.986945 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:06.986904    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:06.995676 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:06.995648    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"]
Jul 01 14:46:06.998251 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:06.998235    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.008030 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.008011    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-kgq6c\""
Jul 01 14:46:07.008114 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.008011    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 14:46:07.008767 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.008750    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 01 14:46:07.008859 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.008764    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 14:46:07.025879 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.025830    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"]
Jul 01 14:46:07.032295 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.032273    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.032295 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.032303    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnqbt\" (UniqueName: \"kubernetes.io/projected/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-kube-api-access-vnqbt\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.071109 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.071083    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:07.133748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.133659    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.133748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.133698    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnqbt\" (UniqueName: \"kubernetes.io/projected/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-kube-api-access-vnqbt\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.134017 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:46:07.133810    2541 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 01 14:46:07.134017 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:46:07.133917    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert podName:a1110e7d-eef9-4ff4-90bf-d8578c0d4597 nodeName:}" failed. No retries permitted until 2026-07-01 14:46:07.633901112 +0000 UTC m=+416.350285846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert") pod "llmisvc-controller-manager-67d95fdc48-zcgz5" (UID: "a1110e7d-eef9-4ff4-90bf-d8578c0d4597") : secret "llmisvc-webhook-server-cert" not found
Jul 01 14:46:07.146269 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.146245    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnqbt\" (UniqueName: \"kubernetes.io/projected/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-kube-api-access-vnqbt\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.153889 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.153861    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:07.266209 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.266180    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:07.637814 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.637781    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.640316 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.640289    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1110e7d-eef9-4ff4-90bf-d8578c0d4597-cert\") pod \"llmisvc-controller-manager-67d95fdc48-zcgz5\" (UID: \"a1110e7d-eef9-4ff4-90bf-d8578c0d4597\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:07.681565 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.681537    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:07.907670 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:07.907638    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:08.038398 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:08.038374    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"]
Jul 01 14:46:08.040298 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:46:08.040190    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda1110e7d_eef9_4ff4_90bf_d8578c0d4597.slice/crio-29475d5ff520485107de43b6cd390625b698b365dd71004b930239b48478ab12 WatchSource:0}: Error finding container 29475d5ff520485107de43b6cd390625b698b365dd71004b930239b48478ab12: Status 404 returned error can't find the container with id 29475d5ff520485107de43b6cd390625b698b365dd71004b930239b48478ab12
Jul 01 14:46:08.130971 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:08.130940    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:08.238228 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:08.238142    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:08.680165 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:08.680127    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5" event={"ID":"a1110e7d-eef9-4ff4-90bf-d8578c0d4597","Type":"ContainerStarted","Data":"29475d5ff520485107de43b6cd390625b698b365dd71004b930239b48478ab12"}
Jul 01 14:46:10.692304 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:10.692266    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5" event={"ID":"a1110e7d-eef9-4ff4-90bf-d8578c0d4597","Type":"ContainerStarted","Data":"32ba05eacb8beca57fc46211adc0ee93d5f5cf929bdc00b52a29b219306c5748"}
Jul 01 14:46:10.692689 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:10.692378    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:33.983902 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:33.983795    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:46:41.699279 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:41.699251    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5"
Jul 01 14:46:41.721322 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:46:41.721254    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-zcgz5" podStartSLOduration=33.727238082 podStartE2EDuration="35.721233897s" totalImagesPullingTime="1.993995815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:46:10.718091649 +0000 UTC m=+419.434476405" watchObservedRunningTime="2026-07-01 14:46:41.721233897 +0000 UTC m=+450.437618656"
Jul 01 14:47:17.125619 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:17.125585    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:17.229592 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:17.229545    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:17.529243 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:17.529211    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:18.278216 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:18.278179    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:18.340860 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:18.340809    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:34.000708 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.000676    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-lkjcs"]
Jul 01 14:47:34.004054 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.004034    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.006743 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.006726    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 01 14:47:34.007260 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.007247    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-kldlb\""
Jul 01 14:47:34.014147 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.014125    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-lkjcs"]
Jul 01 14:47:34.176253 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.176219    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.176426 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.176281    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4blfm\" (UniqueName: \"kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.277247 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.277166    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.277247 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.277232    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4blfm\" (UniqueName: \"kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.277585 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.277568    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.287363 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.287340    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4blfm\" (UniqueName: \"kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm\") pod \"s3-init-lkjcs\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") " pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.328273 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.328243    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-lkjcs"
Jul 01 14:47:34.460573 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:34.460550    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-lkjcs"]
Jul 01 14:47:34.462856 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:47:34.462817    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46f9807a_a638_4af3_bc62_b1c0d4129bff.slice/crio-efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec WatchSource:0}: Error finding container efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec: Status 404 returned error can't find the container with id efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec
Jul 01 14:47:35.074947 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:35.074906    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerStarted","Data":"efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec"}
Jul 01 14:47:39.096163 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:39.096126    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerStarted","Data":"b92642aa3ffe5a843b0d50ae14a7eb4adda015189a17d1a1a16973f313fbd6ba"}
Jul 01 14:47:40.103996 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:40.103895    2541 generic.go:356] "Generic (PLEG): container finished" podID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerID="b92642aa3ffe5a843b0d50ae14a7eb4adda015189a17d1a1a16973f313fbd6ba" exitCode=0
Jul 01 14:47:40.104425 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:40.103993    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerDied","Data":"b92642aa3ffe5a843b0d50ae14a7eb4adda015189a17d1a1a16973f313fbd6ba"}
Jul 01 14:47:42.519753 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:42.519667    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:47:45.133898 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:45.133785    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerStarted","Data":"9b272388cc491d2be02c573c092cf7720add449fff93a1578df8119f366579ef"}
Jul 01 14:47:45.152484 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:47:45.152431    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-lkjcs" podStartSLOduration=2.352404705 podStartE2EDuration="12.152415137s" totalImagesPullingTime="8.232644246s" totalInitContainerRuntime="1.567366186s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:47:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:47:45.151578849 +0000 UTC m=+513.867963619" watchObservedRunningTime="2026-07-01 14:47:45.152415137 +0000 UTC m=+513.868799894"
Jul 01 14:48:23.826299 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:23.826215    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:48:50.442906 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:50.442872    2541 generic.go:356] "Generic (PLEG): container finished" podID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerID="9b272388cc491d2be02c573c092cf7720add449fff93a1578df8119f366579ef" exitCode=0
Jul 01 14:48:50.443278 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:50.442934    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerDied","Data":"9b272388cc491d2be02c573c092cf7720add449fff93a1578df8119f366579ef"}
Jul 01 14:48:51.575029 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.575003    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-lkjcs"
Jul 01 14:48:51.653176 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.653149    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4blfm\" (UniqueName: \"kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm\") pod \"46f9807a-a638-4af3-bc62-b1c0d4129bff\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") "
Jul 01 14:48:51.653342 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.653198    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache\") pod \"46f9807a-a638-4af3-bc62-b1c0d4129bff\" (UID: \"46f9807a-a638-4af3-bc62-b1c0d4129bff\") "
Jul 01 14:48:51.655556 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.655527    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm" (OuterVolumeSpecName: "kube-api-access-4blfm") pod "46f9807a-a638-4af3-bc62-b1c0d4129bff" (UID: "46f9807a-a638-4af3-bc62-b1c0d4129bff"). InnerVolumeSpecName "kube-api-access-4blfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:48:51.672748 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.672704    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache" (OuterVolumeSpecName: "model-cache") pod "46f9807a-a638-4af3-bc62-b1c0d4129bff" (UID: "46f9807a-a638-4af3-bc62-b1c0d4129bff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:48:51.754025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.753991    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4blfm\" (UniqueName: \"kubernetes.io/projected/46f9807a-a638-4af3-bc62-b1c0d4129bff-kube-api-access-4blfm\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:48:51.754025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:51.754022    2541 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46f9807a-a638-4af3-bc62-b1c0d4129bff-model-cache\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:48:52.453134 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:52.453111    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-lkjcs"
Jul 01 14:48:52.453134 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:52.453123    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-lkjcs" event={"ID":"46f9807a-a638-4af3-bc62-b1c0d4129bff","Type":"ContainerDied","Data":"efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec"}
Jul 01 14:48:52.453341 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:52.453156    2541 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efa0375cc2d838e4501984538b9d41787eceda9493bec8ca17790da16da888ec"
Jul 01 14:48:52.833084 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:52.832996    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:48:52.931952 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:48:52.931916    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:04.613909 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.613875    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:04.729950 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.729915    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:49:04.730319 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.730305    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerName="download-hf-model"
Jul 01 14:49:04.730369 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.730321    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerName="download-hf-model"
Jul 01 14:49:04.730369 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.730339    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerName="s3-init"
Jul 01 14:49:04.730369 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.730344    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerName="s3-init"
Jul 01 14:49:04.730458 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.730404    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="46f9807a-a638-4af3-bc62-b1c0d4129bff" containerName="s3-init"
Jul 01 14:49:04.733529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.733511    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.736534 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736510    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 14:49:04.736534 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736529    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 14:49:04.736709 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736522    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 01 14:49:04.736878 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736865    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 14:49:04.736878 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736871    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 14:49:04.737000 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.736882    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9cpsj\""
Jul 01 14:49:04.737385 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.737371    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 01 14:49:04.748964 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.748935    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:49:04.824056 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.824020    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:04.879587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.879505    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.879587 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.879550    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.879771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.879635    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.879771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.879659    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5t2\" (UniqueName: \"kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.879771 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.879701    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.954128 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.954090    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:04.963868 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.961313    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:49:04.966625 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.966603    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:04.969388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.969367    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-680c6-predictor-serving-cert\""
Jul 01 14:49:04.969476 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.969376    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-680c6-kube-rbac-proxy-sar-config\""
Jul 01 14:49:04.975199 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.975173    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:49:04.980350 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980323    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.980478 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980417    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.980478 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980467    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.980645 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:49:04.980515    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 01 14:49:04.980645 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980532    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.980645 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980558    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5t2\" (UniqueName: \"kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.980645 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:49:04.980578    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls podName:bd8c598b-852e-41f9-a396-d2c030df48b7 nodeName:}" failed. No retries permitted until 2026-07-01 14:49:05.480557949 +0000 UTC m=+594.196942710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls") pod "isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 01 14:49:04.981016 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.980994    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.981089 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.981063    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.981333 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.981313    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:04.991312 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:04.991284    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5t2\" (UniqueName: \"kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:05.081495 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.081459    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.081495 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.081500    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.081707 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.081586    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5slx\" (UniqueName: \"kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.084957 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.084909    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.124917 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.124889    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:49:05.128716 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.128701    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.131662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.131608    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 01 14:49:05.131801 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.131779    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 01 14:49:05.141035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.141009    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:49:05.173784 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.173748    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.182230 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.182204    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.182323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.182242    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.182323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.182308    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5slx\" (UniqueName: \"kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.182410 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:49:05.182348    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-680c6-predictor-serving-cert: secret "error-404-isvc-680c6-predictor-serving-cert" not found
Jul 01 14:49:05.182455 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:49:05.182410    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls podName:1a7a1f63-1a0c-40aa-bb5e-2d44e336d230 nodeName:}" failed. No retries permitted until 2026-07-01 14:49:05.682394871 +0000 UTC m=+594.398779609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls") pod "error-404-isvc-680c6-predictor-795547b9b6-hjms6" (UID: "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230") : secret "error-404-isvc-680c6-predictor-serving-cert" not found
Jul 01 14:49:05.182971 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.182955    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.196132 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.196107    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5slx\" (UniqueName: \"kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.245270 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.245240    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.283631 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.283599    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.283770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.283642    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.283770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.283686    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdq5\" (UniqueName: \"kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.283770 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.283721    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.283907 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.283784    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.384475 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.384383    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.384475 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.384468    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.384682 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.384497    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.384682 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.384529    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdq5\" (UniqueName: \"kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.384682 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.384560    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.385396 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.385143    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.385396 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.385287    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.385396 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.385320    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.386162 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.386141    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.387617 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.387595    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.394683 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.394662    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdq5\" (UniqueName: \"kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nnfsp\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.440017 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.439983    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:05.485071 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.485035    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:05.485474 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.485440    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.487606 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.487583    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-l5tzn\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:05.576600 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.576574    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:49:05.578660 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.578636    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:05.579573 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:49:05.579549    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62509ece_58c5_49f6_bdb7_771fd13e70ad.slice/crio-8c7877d29d9d73a18ab3531caebb335a24d1e71e3c339e03e7a785658986d4c0 WatchSource:0}: Error finding container 8c7877d29d9d73a18ab3531caebb335a24d1e71e3c339e03e7a785658986d4c0: Status 404 returned error can't find the container with id 8c7877d29d9d73a18ab3531caebb335a24d1e71e3c339e03e7a785658986d4c0
Jul 01 14:49:05.643942 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.643882    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:05.687034 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.687000    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.690515 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.690486    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") pod \"error-404-isvc-680c6-predictor-795547b9b6-hjms6\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") " pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:05.788725 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.788700    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:49:05.791050 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:49:05.791026    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8c598b_852e_41f9_a396_d2c030df48b7.slice/crio-e5a88831ea6a031c5a542e4f74b6a029e7a06ec2c6ebdba208fc39bff0f01eb1 WatchSource:0}: Error finding container e5a88831ea6a031c5a542e4f74b6a029e7a06ec2c6ebdba208fc39bff0f01eb1: Status 404 returned error can't find the container with id e5a88831ea6a031c5a542e4f74b6a029e7a06ec2c6ebdba208fc39bff0f01eb1
Jul 01 14:49:05.878310 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:05.878282    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:06.026561 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.026537    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:49:06.029464 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:49:06.029422    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7a1f63_1a0c_40aa_bb5e_2d44e336d230.slice/crio-cf4607a28bb9bac4c4cfc7b18c36ed06e542c49562729e16dc28f3a8d89ddde7 WatchSource:0}: Error finding container cf4607a28bb9bac4c4cfc7b18c36ed06e542c49562729e16dc28f3a8d89ddde7: Status 404 returned error can't find the container with id cf4607a28bb9bac4c4cfc7b18c36ed06e542c49562729e16dc28f3a8d89ddde7
Jul 01 14:49:06.518188 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.518148    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerStarted","Data":"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"}
Jul 01 14:49:06.518356 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.518226    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerStarted","Data":"8c7877d29d9d73a18ab3531caebb335a24d1e71e3c339e03e7a785658986d4c0"}
Jul 01 14:49:06.519351 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.519328    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerStarted","Data":"cf4607a28bb9bac4c4cfc7b18c36ed06e542c49562729e16dc28f3a8d89ddde7"}
Jul 01 14:49:06.520618 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.520596    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerStarted","Data":"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"}
Jul 01 14:49:06.520697 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:06.520649    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerStarted","Data":"e5a88831ea6a031c5a542e4f74b6a029e7a06ec2c6ebdba208fc39bff0f01eb1"}
Jul 01 14:49:10.554214 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:10.554180    2541 generic.go:356] "Generic (PLEG): container finished" podID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerID="8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9" exitCode=0
Jul 01 14:49:10.554590 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:10.554253    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerDied","Data":"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"}
Jul 01 14:49:10.556623 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:10.556576    2541 generic.go:356] "Generic (PLEG): container finished" podID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerID="f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812" exitCode=0
Jul 01 14:49:10.556623 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:10.556610    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerDied","Data":"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"}
Jul 01 14:49:11.750746 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:11.750662    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:49:11.755891 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:11.755775    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:49:11.771798 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:11.771776    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:49:11.772573 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:11.772492    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:49:13.024430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:13.024390    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:13.112607 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:13.112571    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:49:34.714246 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:34.714206    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerStarted","Data":"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"}
Jul 01 14:49:34.716718 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:34.716649    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerStarted","Data":"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"}
Jul 01 14:49:34.718976 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:34.718952    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerStarted","Data":"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"}
Jul 01 14:49:37.741005 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.740914    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerStarted","Data":"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"}
Jul 01 14:49:37.741005 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.740995    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:37.741586 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.741029    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:37.742403 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.742365    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:49:37.743631 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.743512    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerStarted","Data":"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"}
Jul 01 14:49:37.743631 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.743590    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:37.746166 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.746138    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerStarted","Data":"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"}
Jul 01 14:49:37.746875 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.746514    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:37.746875 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.746643    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:37.747551 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.747523    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:49:37.762942 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.762897    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podStartSLOduration=2.605654257 podStartE2EDuration="32.762883087s" totalImagesPullingTime="26.457912344s" totalInitContainerRuntime="3.699316486s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:49:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:49:37.761315596 +0000 UTC m=+626.477700354" watchObservedRunningTime="2026-07-01 14:49:37.762883087 +0000 UTC m=+626.479267845"
Jul 01 14:49:37.781809 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.781758    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podStartSLOduration=3.469039555 podStartE2EDuration="33.781743327s" totalImagesPullingTime="26.483257357s" totalInitContainerRuntime="3.829446415s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:49:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:49:37.780603662 +0000 UTC m=+626.496988419" watchObservedRunningTime="2026-07-01 14:49:37.781743327 +0000 UTC m=+626.498128084"
Jul 01 14:49:37.798896 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:37.798823    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podStartSLOduration=10.945574458 podStartE2EDuration="33.798809525s" totalImagesPullingTime="22.853235067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:49:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:49:37.797369277 +0000 UTC m=+626.513754038" watchObservedRunningTime="2026-07-01 14:49:37.798809525 +0000 UTC m=+626.515194281"
Jul 01 14:49:38.757074 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:38.757030    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:49:38.757537 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:38.757156    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:49:38.757537 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:38.757302    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:38.758498 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:38.758475    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:49:39.762141 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:39.762100    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:49:39.762534 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:39.762100    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:49:43.761914 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:43.761886    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:49:43.762506 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:43.762477    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:49:44.766363 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:44.766333    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:49:44.766759 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:44.766545    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:49:44.766863 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:44.766802    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:49:44.767221 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:44.767199    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:49:53.762795 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:53.762758    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:49:54.766855 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:54.766801    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:49:54.767309 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:49:54.767272    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:50:03.763091 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:03.763051    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:50:04.767187 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:04.767151    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:50:04.767567 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:04.767284    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:50:13.763341 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:13.763302    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:50:14.767619 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:14.767583    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 01 14:50:14.767619 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:14.767607    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:50:23.763220 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:23.763183    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:50:24.766769 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:24.766727    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:50:24.767753 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:24.767736    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:50:24.908730 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:24.908696    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:33.762570 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:33.762531    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 01 14:50:34.639336 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:34.639298    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:34.767569 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:34.767530    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:50:34.899676 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:34.899589    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:43.763727 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:43.763693    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:50:44.767434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:44.767394    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 01 14:50:44.774610 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:44.774582    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:44.948456 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:44.948428    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:45.098355 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.098272    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:45.133979 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.133951    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 14:50:45.137756 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.137731    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.140684 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.140661    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3297d-predictor-serving-cert\""
Jul 01 14:50:45.140822 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.140661    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3297d-kube-rbac-proxy-sar-config\""
Jul 01 14:50:45.153000 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.152974    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 14:50:45.194224 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.194192    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:45.232663 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.232630    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:50:45.232936 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.232911    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container" containerID="cri-o://07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad" gracePeriod=30
Jul 01 14:50:45.233033 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.232962    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kube-rbac-proxy" containerID="cri-o://f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d" gracePeriod=30
Jul 01 14:50:45.257400 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.257376    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:50:45.309525 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.309491    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wqs2\" (UniqueName: \"kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.309725 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.309533    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.309725 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.309553    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.410620 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.410591    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wqs2\" (UniqueName: \"kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.410775 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.410629    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.410775 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.410651    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.411313 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.411294    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.413138 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.413119    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.420561 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.420536    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wqs2\" (UniqueName: \"kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2\") pod \"error-404-isvc-3297d-predictor-647cb88f78-7q4rr\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") " pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.448375 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.448349    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:45.585689 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:45.585667    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 14:50:45.587931 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:50:45.587902    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa66ef8_22f0_4980_b0b8_adba388ec0d1.slice/crio-70c172f7bbe65729fcd92c74aa7f49ffd061bff16d0e39ed68e2e61354682865 WatchSource:0}: Error finding container 70c172f7bbe65729fcd92c74aa7f49ffd061bff16d0e39ed68e2e61354682865: Status 404 returned error can't find the container with id 70c172f7bbe65729fcd92c74aa7f49ffd061bff16d0e39ed68e2e61354682865
Jul 01 14:50:46.087561 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.087464    2541 generic.go:356] "Generic (PLEG): container finished" podID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerID="f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d" exitCode=2
Jul 01 14:50:46.087561 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.087539    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerDied","Data":"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"}
Jul 01 14:50:46.089314 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.089290    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerStarted","Data":"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"}
Jul 01 14:50:46.089434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.089344    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerStarted","Data":"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"}
Jul 01 14:50:46.089434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.089381    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:46.089434 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.089410    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerStarted","Data":"70c172f7bbe65729fcd92c74aa7f49ffd061bff16d0e39ed68e2e61354682865"}
Jul 01 14:50:46.089607 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.089537    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:46.090730 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.090703    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:50:46.114248 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:46.114202    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podStartSLOduration=1.114187878 podStartE2EDuration="1.114187878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:50:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:50:46.112494229 +0000 UTC m=+694.828878987" watchObservedRunningTime="2026-07-01 14:50:46.114187878 +0000 UTC m=+694.830572633"
Jul 01 14:50:47.094489 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:47.094453    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:50:48.589165 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.589107    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:50:48.741668 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.741623    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5slx\" (UniqueName: \"kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx\") pod \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") "
Jul 01 14:50:48.741897 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.741692    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") pod \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") "
Jul 01 14:50:48.741897 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.741728    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config\") pod \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\" (UID: \"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230\") "
Jul 01 14:50:48.742169 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.742142    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" (UID: "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:50:48.744061 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.744034    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx" (OuterVolumeSpecName: "kube-api-access-w5slx") pod "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" (UID: "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230"). InnerVolumeSpecName "kube-api-access-w5slx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:50:48.744061 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.744049    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" (UID: "1a7a1f63-1a0c-40aa-bb5e-2d44e336d230"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:50:48.843149 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.842935    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:50:48.843149 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.842971    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5slx\" (UniqueName: \"kubernetes.io/projected/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-kube-api-access-w5slx\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:50:48.843149 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:48.842986    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:50:49.105764 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.105686    2541 generic.go:356] "Generic (PLEG): container finished" podID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerID="07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad" exitCode=0
Jul 01 14:50:49.105764 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.105749    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"
Jul 01 14:50:49.106033 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.105774    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerDied","Data":"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"}
Jul 01 14:50:49.106033 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.105874    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6" event={"ID":"1a7a1f63-1a0c-40aa-bb5e-2d44e336d230","Type":"ContainerDied","Data":"cf4607a28bb9bac4c4cfc7b18c36ed06e542c49562729e16dc28f3a8d89ddde7"}
Jul 01 14:50:49.106033 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.105913    2541 scope.go:117] "RemoveContainer" containerID="f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"
Jul 01 14:50:49.115445 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.115423    2541 scope.go:117] "RemoveContainer" containerID="07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"
Jul 01 14:50:49.124723 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.124699    2541 scope.go:117] "RemoveContainer" containerID="f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"
Jul 01 14:50:49.125061 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:50:49.125034    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d\": container with ID starting with f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d not found: ID does not exist" containerID="f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"
Jul 01 14:50:49.125136 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.125074    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d"} err="failed to get container status \"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d\": rpc error: code = NotFound desc = could not find container \"f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d\": container with ID starting with f3fe888a26898e10ae9737b1a2e590fc19e9f8080c25af4cea2563866f5d753d not found: ID does not exist"
Jul 01 14:50:49.125136 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.125102    2541 scope.go:117] "RemoveContainer" containerID="07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"
Jul 01 14:50:49.125397 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:50:49.125376    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad\": container with ID starting with 07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad not found: ID does not exist" containerID="07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"
Jul 01 14:50:49.125474 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.125407    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad"} err="failed to get container status \"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad\": rpc error: code = NotFound desc = could not find container \"07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad\": container with ID starting with 07e5547d2f5322110c5a50a431b94f33289fa438e4f6a3e93718eccf150915ad not found: ID does not exist"
Jul 01 14:50:49.129401 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.129374    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:50:49.134008 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.133979    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-680c6-predictor-795547b9b6-hjms6"]
Jul 01 14:50:49.725603 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:49.725571    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" path="/var/lib/kubelet/pods/1a7a1f63-1a0c-40aa-bb5e-2d44e336d230/volumes"
Jul 01 14:50:52.099678 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:52.099589    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:50:52.100173 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:52.100140    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:50:54.767443 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:54.767413    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:50:58.228547 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:50:58.228511    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:02.101035 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:02.100943    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:51:12.100859 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:12.100756    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:51:22.100703 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:22.100662    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 14:51:26.394087 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:26.394055    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:26.454804 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:26.454777    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:32.100742 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:32.100710    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 14:51:34.825755 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:34.825726    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:34.951429 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:34.951399    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:51:34.951684 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:34.951659    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container" containerID="cri-o://d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5" gracePeriod=30
Jul 01 14:51:34.951746 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:34.951694    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kube-rbac-proxy" containerID="cri-o://3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045" gracePeriod=30
Jul 01 14:51:34.954701 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:34.954675    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.034286 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.034257    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.084085 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084014    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 14:51:35.084415 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084403    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kube-rbac-proxy"
Jul 01 14:51:35.084463 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084417    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kube-rbac-proxy"
Jul 01 14:51:35.084463 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084430    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container"
Jul 01 14:51:35.084463 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084435    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container"
Jul 01 14:51:35.084578 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084500    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kserve-container"
Jul 01 14:51:35.084578 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.084512    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a7a1f63-1a0c-40aa-bb5e-2d44e336d230" containerName="kube-rbac-proxy"
Jul 01 14:51:35.087505 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.087489    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.089934 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.089916    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0d08f-predictor-serving-cert\""
Jul 01 14:51:35.090025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.089918    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0d08f-kube-rbac-proxy-sar-config\""
Jul 01 14:51:35.099594 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.099572    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 14:51:35.113654 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.113627    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.165323 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.165295    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:51:35.165556 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.165534    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container" containerID="cri-o://8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d" gracePeriod=30
Jul 01 14:51:35.165631 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.165581    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kube-rbac-proxy" containerID="cri-o://cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e" gracePeriod=30
Jul 01 14:51:35.170918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.170815    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.170918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.170905    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xmhd\" (UniqueName: \"kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.171088 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.170938    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.188183 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.188153    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.272245 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.272199    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.272425 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.272260    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xmhd\" (UniqueName: \"kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.272425 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.272287    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.272544 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:35.272444    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-serving-cert: secret "error-404-isvc-0d08f-predictor-serving-cert" not found
Jul 01 14:51:35.272544 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:35.272521    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls podName:407379a1-739e-4c98-8bc7-8e2a23fb15e1 nodeName:}" failed. No retries permitted until 2026-07-01 14:51:35.772504351 +0000 UTC m=+744.488889088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls") pod "error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" (UID: "407379a1-739e-4c98-8bc7-8e2a23fb15e1") : secret "error-404-isvc-0d08f-predictor-serving-cert" not found
Jul 01 14:51:35.272931 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.272913    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.281361 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.281336    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xmhd\" (UniqueName: \"kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.286041 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.286013    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.337803 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.337713    2541 generic.go:356] "Generic (PLEG): container finished" podID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerID="cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e" exitCode=2
Jul 01 14:51:35.337803 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.337787    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerDied","Data":"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"}
Jul 01 14:51:35.340121 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.340098    2541 generic.go:356] "Generic (PLEG): container finished" podID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerID="3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045" exitCode=2
Jul 01 14:51:35.340269 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.340130    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerDied","Data":"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"}
Jul 01 14:51:35.385360 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.385334    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.520750 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.520719    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:35.775607 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.775567    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.778194 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.778163    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") pod \"error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") " pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:35.998326 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:35.998283    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:36.140411 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.140386    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 14:51:36.143658 ip-10-0-133-214 kubenswrapper[2541]: W0701 14:51:36.143609    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407379a1_739e_4c98_8bc7_8e2a23fb15e1.slice/crio-af8b3ecd21df5b24f2823e158402147a379dceb2882526248d944cbb2f6af3c4 WatchSource:0}: Error finding container af8b3ecd21df5b24f2823e158402147a379dceb2882526248d944cbb2f6af3c4: Status 404 returned error can't find the container with id af8b3ecd21df5b24f2823e158402147a379dceb2882526248d944cbb2f6af3c4
Jul 01 14:51:36.349202 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.349114    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerStarted","Data":"9f4bcf7ece1a03d19ba8fabe93c556f808b96cd1d001aed334ad6ecb7fb8e4d7"}
Jul 01 14:51:36.349202 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.349183    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerStarted","Data":"61e0e894e88dbb09ed4459752392b9c06d63c0168232c43189a25afd5c66f442"}
Jul 01 14:51:36.349405 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.349221    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:36.349405 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.349246    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerStarted","Data":"af8b3ecd21df5b24f2823e158402147a379dceb2882526248d944cbb2f6af3c4"}
Jul 01 14:51:36.375140 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:36.375082    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podStartSLOduration=1.37506524 podStartE2EDuration="1.37506524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:51:36.373292213 +0000 UTC m=+745.089676967" watchObservedRunningTime="2026-07-01 14:51:36.37506524 +0000 UTC m=+745.091449997"
Jul 01 14:51:37.354492 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:37.354463    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:37.355866 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:37.355821    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:51:38.360430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:38.360390    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:51:38.757278 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:38.757243    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 01 14:51:38.920956 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:38.920931    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:51:39.008490 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008390    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert\") pod \"62509ece-58c5-49f6-bdb7-771fd13e70ad\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") "
Jul 01 14:51:39.008490 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008444    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location\") pod \"62509ece-58c5-49f6-bdb7-771fd13e70ad\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") "
Jul 01 14:51:39.008490 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008476    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config\") pod \"62509ece-58c5-49f6-bdb7-771fd13e70ad\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") "
Jul 01 14:51:39.008793 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008526    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls\") pod \"62509ece-58c5-49f6-bdb7-771fd13e70ad\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") "
Jul 01 14:51:39.008793 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008548    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzdq5\" (UniqueName: \"kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5\") pod \"62509ece-58c5-49f6-bdb7-771fd13e70ad\" (UID: \"62509ece-58c5-49f6-bdb7-771fd13e70ad\") "
Jul 01 14:51:39.008939 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008832    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "62509ece-58c5-49f6-bdb7-771fd13e70ad" (UID: "62509ece-58c5-49f6-bdb7-771fd13e70ad"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:51:39.008939 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008874    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "62509ece-58c5-49f6-bdb7-771fd13e70ad" (UID: "62509ece-58c5-49f6-bdb7-771fd13e70ad"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:39.008939 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.008916    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "62509ece-58c5-49f6-bdb7-771fd13e70ad" (UID: "62509ece-58c5-49f6-bdb7-771fd13e70ad"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:39.010888 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.010866    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5" (OuterVolumeSpecName: "kube-api-access-hzdq5") pod "62509ece-58c5-49f6-bdb7-771fd13e70ad" (UID: "62509ece-58c5-49f6-bdb7-771fd13e70ad"). InnerVolumeSpecName "kube-api-access-hzdq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:51:39.010975 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.010887    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "62509ece-58c5-49f6-bdb7-771fd13e70ad" (UID: "62509ece-58c5-49f6-bdb7-771fd13e70ad"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:51:39.109754 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.109716    2541 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-cabundle-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:39.109754 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.109750    2541 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/62509ece-58c5-49f6-bdb7-771fd13e70ad-kserve-provision-location\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:39.109754 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.109760    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:39.110025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.109770    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62509ece-58c5-49f6-bdb7-771fd13e70ad-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:39.110025 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.109778    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzdq5\" (UniqueName: \"kubernetes.io/projected/62509ece-58c5-49f6-bdb7-771fd13e70ad-kube-api-access-hzdq5\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:39.366078 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.365998    2541 generic.go:356] "Generic (PLEG): container finished" podID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerID="8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d" exitCode=0
Jul 01 14:51:39.366078 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.366044    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerDied","Data":"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"}
Jul 01 14:51:39.366078 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.366053    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"
Jul 01 14:51:39.366553 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.366119    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp" event={"ID":"62509ece-58c5-49f6-bdb7-771fd13e70ad","Type":"ContainerDied","Data":"8c7877d29d9d73a18ab3531caebb335a24d1e71e3c339e03e7a785658986d4c0"}
Jul 01 14:51:39.366553 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.366145    2541 scope.go:117] "RemoveContainer" containerID="cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"
Jul 01 14:51:39.375683 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.375661    2541 scope.go:117] "RemoveContainer" containerID="8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"
Jul 01 14:51:39.383768 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.383752    2541 scope.go:117] "RemoveContainer" containerID="8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"
Jul 01 14:51:39.390521 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.390499    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:51:39.392367 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.392347    2541 scope.go:117] "RemoveContainer" containerID="cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"
Jul 01 14:51:39.392619 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:39.392602    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e\": container with ID starting with cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e not found: ID does not exist" containerID="cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"
Jul 01 14:51:39.392686 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.392627    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e"} err="failed to get container status \"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e\": rpc error: code = NotFound desc = could not find container \"cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e\": container with ID starting with cf296012d3608db1797010d8781949f0ce366b8da0e0b3f77398b89664c9ac9e not found: ID does not exist"
Jul 01 14:51:39.392686 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.392645    2541 scope.go:117] "RemoveContainer" containerID="8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"
Jul 01 14:51:39.392897 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:39.392879    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d\": container with ID starting with 8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d not found: ID does not exist" containerID="8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"
Jul 01 14:51:39.392950 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.392902    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d"} err="failed to get container status \"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d\": rpc error: code = NotFound desc = could not find container \"8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d\": container with ID starting with 8f73f91a3703a842e257877c64d7dbb8fef94784e617fe2cf0bc772a2b096b5d not found: ID does not exist"
Jul 01 14:51:39.392950 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.392916    2541 scope.go:117] "RemoveContainer" containerID="8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"
Jul 01 14:51:39.393153 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:39.393135    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9\": container with ID starting with 8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9 not found: ID does not exist" containerID="8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"
Jul 01 14:51:39.393199 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.393159    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9"} err="failed to get container status \"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9\": rpc error: code = NotFound desc = could not find container \"8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9\": container with ID starting with 8bb2234b2a2723ec82877f1c729f17829632f06587851a1185b16c4f662694e9 not found: ID does not exist"
Jul 01 14:51:39.401303 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.401279    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nnfsp"]
Jul 01 14:51:39.725005 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.724976    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" path="/var/lib/kubelet/pods/62509ece-58c5-49f6-bdb7-771fd13e70ad/volumes"
Jul 01 14:51:39.762179 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.762148    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 01 14:51:39.906888 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:39.906865    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:51:40.017430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017334    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert\") pod \"bd8c598b-852e-41f9-a396-d2c030df48b7\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") "
Jul 01 14:51:40.017430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017369    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x5t2\" (UniqueName: \"kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2\") pod \"bd8c598b-852e-41f9-a396-d2c030df48b7\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") "
Jul 01 14:51:40.017430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017401    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location\") pod \"bd8c598b-852e-41f9-a396-d2c030df48b7\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") "
Jul 01 14:51:40.017430 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017427    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config\") pod \"bd8c598b-852e-41f9-a396-d2c030df48b7\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") "
Jul 01 14:51:40.017792 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017491    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") pod \"bd8c598b-852e-41f9-a396-d2c030df48b7\" (UID: \"bd8c598b-852e-41f9-a396-d2c030df48b7\") "
Jul 01 14:51:40.017792 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017752    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "bd8c598b-852e-41f9-a396-d2c030df48b7" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:40.017792 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017768    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "bd8c598b-852e-41f9-a396-d2c030df48b7" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:51:40.017963 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.017798    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bd8c598b-852e-41f9-a396-d2c030df48b7" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:40.019628 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.019598    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2" (OuterVolumeSpecName: "kube-api-access-8x5t2") pod "bd8c598b-852e-41f9-a396-d2c030df48b7" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7"). InnerVolumeSpecName "kube-api-access-8x5t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:51:40.019738 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.019655    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bd8c598b-852e-41f9-a396-d2c030df48b7" (UID: "bd8c598b-852e-41f9-a396-d2c030df48b7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:51:40.118603 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.118573    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd8c598b-852e-41f9-a396-d2c030df48b7-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.118603 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.118600    2541 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-cabundle-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.118603 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.118608    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x5t2\" (UniqueName: \"kubernetes.io/projected/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-api-access-8x5t2\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.118830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.118618    2541 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd8c598b-852e-41f9-a396-d2c030df48b7-kserve-provision-location\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.118830 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.118627    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd8c598b-852e-41f9-a396-d2c030df48b7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.373290 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.373201    2541 generic.go:356] "Generic (PLEG): container finished" podID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerID="d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5" exitCode=0
Jul 01 14:51:40.373290 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.373243    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerDied","Data":"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"}
Jul 01 14:51:40.373290 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.373260    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"
Jul 01 14:51:40.373803 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.373305    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn" event={"ID":"bd8c598b-852e-41f9-a396-d2c030df48b7","Type":"ContainerDied","Data":"e5a88831ea6a031c5a542e4f74b6a029e7a06ec2c6ebdba208fc39bff0f01eb1"}
Jul 01 14:51:40.373803 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.373341    2541 scope.go:117] "RemoveContainer" containerID="3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"
Jul 01 14:51:40.382882 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.382862    2541 scope.go:117] "RemoveContainer" containerID="d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"
Jul 01 14:51:40.392584 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.392566    2541 scope.go:117] "RemoveContainer" containerID="f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"
Jul 01 14:51:40.400781 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.400760    2541 scope.go:117] "RemoveContainer" containerID="3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"
Jul 01 14:51:40.401069 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:40.401051    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045\": container with ID starting with 3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045 not found: ID does not exist" containerID="3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"
Jul 01 14:51:40.401146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.401077    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045"} err="failed to get container status \"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045\": rpc error: code = NotFound desc = could not find container \"3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045\": container with ID starting with 3af6ecc45e95d10f28ba4ca3d801c254064b28157006135ae7e3111d5243b045 not found: ID does not exist"
Jul 01 14:51:40.401146 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.401095    2541 scope.go:117] "RemoveContainer" containerID="d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"
Jul 01 14:51:40.401308 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:40.401291    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5\": container with ID starting with d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5 not found: ID does not exist" containerID="d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"
Jul 01 14:51:40.401368 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.401311    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5"} err="failed to get container status \"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5\": rpc error: code = NotFound desc = could not find container \"d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5\": container with ID starting with d122ac475f054a94ef7e768788e402d8694225eadb9308d61768c56ece99feb5 not found: ID does not exist"
Jul 01 14:51:40.401368 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.401325    2541 scope.go:117] "RemoveContainer" containerID="f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"
Jul 01 14:51:40.401580 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:51:40.401556    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812\": container with ID starting with f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812 not found: ID does not exist" containerID="f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"
Jul 01 14:51:40.401638 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.401585    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812"} err="failed to get container status \"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812\": rpc error: code = NotFound desc = could not find container \"f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812\": container with ID starting with f56cdb629c6a58c49e0be8363cbfe418aa434fe85d0473f8049926a086156812 not found: ID does not exist"
Jul 01 14:51:40.402623 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.402599    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:51:40.422102 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:40.422077    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-l5tzn"]
Jul 01 14:51:41.725079 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:41.725052    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" path="/var/lib/kubelet/pods/bd8c598b-852e-41f9-a396-d2c030df48b7/volumes"
Jul 01 14:51:43.365616 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:43.365584    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:51:43.366057 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:43.366001    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:51:45.010532 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:45.010504    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:45.448258 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:45.448225    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:51:53.366263 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:51:53.366218    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:52:03.366213 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:52:03.366165    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:52:13.366541 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:52:13.366501    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 01 14:52:23.366662 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:52:23.366634    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 14:52:45.000060 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:52:44.999964    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:52:45.285049 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:52:45.284967    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:54:09.152547 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:54:09.152467    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:54:11.809505 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:54:11.809473    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:54:11.811863 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:54:11.811816    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:54:11.814111 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:54:11.814091    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:54:11.816363 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:54:11.816343    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:55:21.136202 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:55:21.136155    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:56:10.069460 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:56:10.069428    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:57:23.459918 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:23.459881    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:57:23.531560 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:23.531521    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:57:34.761345 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:34.761299    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:57:34.931219 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:34.931184    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:57:35.031391 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:35.031310    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:57:59.785501 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:57:59.785458    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67c54b9b7c-vtlsd" podUID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" containerName="console" containerID="cri-o://b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717" gracePeriod=15
Jul 01 14:58:00.035411 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.035381    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:58:00.136604 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136501    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136604 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136573    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136604 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136614    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136646    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136707    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136733    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk79w\" (UniqueName: \"kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.136887 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.136776    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle\") pod \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\" (UID: \"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e\") "
Jul 01 14:58:00.137116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137004    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:58:00.137116 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137013    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:58:00.137233 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137131    2541 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-oauth-serving-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.137233 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137150    2541 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-service-ca\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.137340 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137139    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config" (OuterVolumeSpecName: "console-config") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:58:00.137340 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.137320    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:58:00.139010 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.138989    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:58:00.139150 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.139030    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:58:00.139150 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.139049    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w" (OuterVolumeSpecName: "kube-api-access-nk79w") pod "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" (UID: "8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e"). InnerVolumeSpecName "kube-api-access-nk79w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:58:00.167362 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.167333    2541 generic.go:356] "Generic (PLEG): container finished" podID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" containerID="b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717" exitCode=0
Jul 01 14:58:00.167529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.167381    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c54b9b7c-vtlsd"
Jul 01 14:58:00.167529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.167417    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c54b9b7c-vtlsd" event={"ID":"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e","Type":"ContainerDied","Data":"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"}
Jul 01 14:58:00.167529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.167464    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c54b9b7c-vtlsd" event={"ID":"8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e","Type":"ContainerDied","Data":"a06bd95ae3840f33cdf7b8f841a87cff779e860cdde61243eb50a6f5f15efb94"}
Jul 01 14:58:00.167529 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.167489    2541 scope.go:117] "RemoveContainer" containerID="b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"
Jul 01 14:58:00.177058 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.177040    2541 scope.go:117] "RemoveContainer" containerID="b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"
Jul 01 14:58:00.177349 ip-10-0-133-214 kubenswrapper[2541]: E0701 14:58:00.177330    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717\": container with ID starting with b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717 not found: ID does not exist" containerID="b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"
Jul 01 14:58:00.177388 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.177359    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717"} err="failed to get container status \"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717\": rpc error: code = NotFound desc = could not find container \"b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717\": container with ID starting with b9224ad1e55834b1927082fd33e10a2deb0b98f62242135f84ec5a2cb0a33717 not found: ID does not exist"
Jul 01 14:58:00.188889 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.188858    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:58:00.192365 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.192343    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67c54b9b7c-vtlsd"]
Jul 01 14:58:00.238331 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.238301    2541 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-trusted-ca-bundle\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.238331 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.238326    2541 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.238331 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.238339    2541 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-serving-cert\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.238543 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.238348    2541 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-console-oauth-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:00.238543 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:00.238357    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk79w\" (UniqueName: \"kubernetes.io/projected/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e-kube-api-access-nk79w\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 14:58:01.724365 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:58:01.724329    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" path="/var/lib/kubelet/pods/8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e/volumes"
Jul 01 14:59:11.838540 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:11.838511    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:59:11.842464 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:11.842442    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 14:59:11.843054 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:11.843037    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:59:11.847112 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:11.847085    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 14:59:59.811881 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:59.811832    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 14:59:59.928512 ip-10-0-133-214 kubenswrapper[2541]: I0701 14:59:59.928477    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:00.011547 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.011518    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:00.056023 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.055990    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:00:00.056397 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056386    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056439 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056400    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056439 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056422    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056439 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056428    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056439 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056436    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" containerName="console"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056441    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" containerName="console"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056447    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056452    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056462    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="storage-initializer"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056468    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="storage-initializer"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056480    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="storage-initializer"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056485    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="storage-initializer"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056493    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056504    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056561    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bdc8cc3-2882-4471-8ef9-25f9db7c5c0e" containerName="console"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056569    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056574    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kube-rbac-proxy"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056580    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8c598b-852e-41f9-a396-d2c030df48b7" containerName="kserve-container"
Jul 01 15:00:00.056601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.056588    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="62509ece-58c5-49f6-bdb7-771fd13e70ad" containerName="kserve-container"
Jul 01 15:00:00.059704 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.059688    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.062229 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.062173    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7bec4-kube-rbac-proxy-sar-config\""
Jul 01 15:00:00.062349 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.062248    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7bec4-predictor-serving-cert\""
Jul 01 15:00:00.070802 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.070766    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:00:00.087444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.087410    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:00.095729 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.095703    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 15:00:00.095976 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.095934    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" containerID="cri-o://f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d" gracePeriod=30
Jul 01 15:00:00.096076 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.095963    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kube-rbac-proxy" containerID="cri-o://8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d" gracePeriod=30
Jul 01 15:00:00.187715 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.187671    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpvdz\" (UniqueName: \"kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.187915 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.187724    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.187915 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.187750    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.190940 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.190914    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:00.285101 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.285069    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:00.288283 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.288263    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.288366 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.288300    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.288404 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.288382    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpvdz\" (UniqueName: \"kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.289120 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.289027    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.290938 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.290917    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.296542 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.296523    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpvdz\" (UniqueName: \"kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz\") pod \"error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") " pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.371392 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.371301    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.511478 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.511454    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:00:00.513737 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:00:00.513709    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fb3c58_5c38_4faa_99bf_844a1ab8a9de.slice/crio-88cc49b99fe251491c274e721956ba020eadf7db0bac0ee6a535491dc41385af WatchSource:0}: Error finding container 88cc49b99fe251491c274e721956ba020eadf7db0bac0ee6a535491dc41385af: Status 404 returned error can't find the container with id 88cc49b99fe251491c274e721956ba020eadf7db0bac0ee6a535491dc41385af
Jul 01 15:00:00.744291 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.744252    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerStarted","Data":"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"}
Jul 01 15:00:00.744514 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.744494    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.744605 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.744550    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerStarted","Data":"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"}
Jul 01 15:00:00.744605 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.744585    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerStarted","Data":"88cc49b99fe251491c274e721956ba020eadf7db0bac0ee6a535491dc41385af"}
Jul 01 15:00:00.744722 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.744617    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:00.745910 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.745878    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:00.746339 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.746321    2541 generic.go:356] "Generic (PLEG): container finished" podID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerID="8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d" exitCode=2
Jul 01 15:00:00.746420 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.746364    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerDied","Data":"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"}
Jul 01 15:00:00.765348 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:00.765299    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podStartSLOduration=0.765284829 podStartE2EDuration="765.284829ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:00:00.764090118 +0000 UTC m=+1249.480474868" watchObservedRunningTime="2026-07-01 15:00:00.765284829 +0000 UTC m=+1249.481669584"
Jul 01 15:00:01.751765 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:01.751727    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:02.095407 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:02.095287    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 15:00:02.100717 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:02.100692    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 15:00:03.454261 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.454238    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 15:00:03.618601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.618498    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wqs2\" (UniqueName: \"kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2\") pod \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") "
Jul 01 15:00:03.618601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.618600    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config\") pod \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") "
Jul 01 15:00:03.618807 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.618690    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls\") pod \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\" (UID: \"efa66ef8-22f0-4980-b0b8-adba388ec0d1\") "
Jul 01 15:00:03.618972 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.618950    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "efa66ef8-22f0-4980-b0b8-adba388ec0d1" (UID: "efa66ef8-22f0-4980-b0b8-adba388ec0d1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:00:03.619061 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.619047    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:00:03.620799 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.620777    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "efa66ef8-22f0-4980-b0b8-adba388ec0d1" (UID: "efa66ef8-22f0-4980-b0b8-adba388ec0d1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:00:03.620928 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.620913    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2" (OuterVolumeSpecName: "kube-api-access-6wqs2") pod "efa66ef8-22f0-4980-b0b8-adba388ec0d1" (UID: "efa66ef8-22f0-4980-b0b8-adba388ec0d1"). InnerVolumeSpecName "kube-api-access-6wqs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:00:03.720498 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.720458    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wqs2\" (UniqueName: \"kubernetes.io/projected/efa66ef8-22f0-4980-b0b8-adba388ec0d1-kube-api-access-6wqs2\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:00:03.720498 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.720489    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa66ef8-22f0-4980-b0b8-adba388ec0d1-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:00:03.763038 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.763009    2541 generic.go:356] "Generic (PLEG): container finished" podID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerID="f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d" exitCode=0
Jul 01 15:00:03.763204 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.763060    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"
Jul 01 15:00:03.763204 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.763062    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerDied","Data":"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"}
Jul 01 15:00:03.763294 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.763198    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr" event={"ID":"efa66ef8-22f0-4980-b0b8-adba388ec0d1","Type":"ContainerDied","Data":"70c172f7bbe65729fcd92c74aa7f49ffd061bff16d0e39ed68e2e61354682865"}
Jul 01 15:00:03.763294 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.763237    2541 scope.go:117] "RemoveContainer" containerID="8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"
Jul 01 15:00:03.774064 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.774028    2541 scope.go:117] "RemoveContainer" containerID="f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"
Jul 01 15:00:03.781822 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.781801    2541 scope.go:117] "RemoveContainer" containerID="8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"
Jul 01 15:00:03.782191 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:00:03.782161    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d\": container with ID starting with 8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d not found: ID does not exist" containerID="8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"
Jul 01 15:00:03.782423 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.782201    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d"} err="failed to get container status \"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d\": rpc error: code = NotFound desc = could not find container \"8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d\": container with ID starting with 8a3758e8c44d240573b1493ad56a4ce0a0d2966a5e24c81767c42cfa9fa72e0d not found: ID does not exist"
Jul 01 15:00:03.782423 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.782226    2541 scope.go:117] "RemoveContainer" containerID="f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"
Jul 01 15:00:03.782656 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:00:03.782640    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d\": container with ID starting with f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d not found: ID does not exist" containerID="f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"
Jul 01 15:00:03.782695 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.782663    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d"} err="failed to get container status \"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d\": rpc error: code = NotFound desc = could not find container \"f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d\": container with ID starting with f678656a0e2ff73f1ad7f2d724834272b056e0aa93d95db685a20b49cb0c708d not found: ID does not exist"
Jul 01 15:00:03.784190 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.784173    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 15:00:03.787803 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:03.787783    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3297d-predictor-647cb88f78-7q4rr"]
Jul 01 15:00:05.724480 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:05.724443    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" path="/var/lib/kubelet/pods/efa66ef8-22f0-4980-b0b8-adba388ec0d1/volumes"
Jul 01 15:00:06.757795 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:06.757767    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:06.758241 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:06.758215    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:16.758955 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:16.758917    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:26.759042 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:26.758999    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:36.758425 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:36.758386    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 15:00:46.759438 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:46.759411    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:00:59.670096 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.670063    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:00:59.899320 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899286    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:00:59.899686 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899672    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.899734 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899688    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.899734 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899703    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container"
Jul 01 15:00:59.899734 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899709    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container"
Jul 01 15:00:59.899830 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899778    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.899830 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.899786    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa66ef8-22f0-4980-b0b8-adba388ec0d1" containerName="kserve-container"
Jul 01 15:00:59.903169 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.903151    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:00:59.905577 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.905550    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-102b8-predictor-serving-cert\""
Jul 01 15:00:59.905730 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.905632    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-102b8-kube-rbac-proxy-sar-config\""
Jul 01 15:00:59.911756 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.911731    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:00:59.942473 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.942369    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 15:00:59.942704 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.942662    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container" containerID="cri-o://61e0e894e88dbb09ed4459752392b9c06d63c0168232c43189a25afd5c66f442" gracePeriod=30
Jul 01 15:00:59.942783 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:00:59.942711    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kube-rbac-proxy" containerID="cri-o://9f4bcf7ece1a03d19ba8fabe93c556f808b96cd1d001aed334ad6ecb7fb8e4d7" gracePeriod=30
Jul 01 15:01:00.012619 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.012582    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.012806 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.012637    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.012806 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.012686    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsdct\" (UniqueName: \"kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.113700 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.113668    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.113903 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.113712    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.113903 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.113731    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsdct\" (UniqueName: \"kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.113903 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:01:00.113870    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-102b8-predictor-serving-cert: secret "error-404-isvc-102b8-predictor-serving-cert" not found
Jul 01 15:01:00.114040 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:01:00.113958    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls podName:0376aec0-5160-4084-8722-f11dd701e097 nodeName:}" failed. No retries permitted until 2026-07-01 15:01:00.613935512 +0000 UTC m=+1309.330320270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls") pod "error-404-isvc-102b8-predictor-548dfbff4-mm86m" (UID: "0376aec0-5160-4084-8722-f11dd701e097") : secret "error-404-isvc-102b8-predictor-serving-cert" not found
Jul 01 15:01:00.114337 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.114320    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.124624 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.124595    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsdct\" (UniqueName: \"kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.618653 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.618611    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.621175 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.621144    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") pod \"error-404-isvc-102b8-predictor-548dfbff4-mm86m\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") " pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.815042 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.815001    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:00.954311 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:00.954280    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:01:00.957730 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:01:00.957702    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0376aec0_5160_4084_8722_f11dd701e097.slice/crio-4dd56c3a6d70f24f1f420b146fcbad4aaed8c4f1ecc4af143d01b24a58f6a6a5 WatchSource:0}: Error finding container 4dd56c3a6d70f24f1f420b146fcbad4aaed8c4f1ecc4af143d01b24a58f6a6a5: Status 404 returned error can't find the container with id 4dd56c3a6d70f24f1f420b146fcbad4aaed8c4f1ecc4af143d01b24a58f6a6a5
Jul 01 15:01:01.047601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:01.047569    2541 generic.go:356] "Generic (PLEG): container finished" podID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerID="9f4bcf7ece1a03d19ba8fabe93c556f808b96cd1d001aed334ad6ecb7fb8e4d7" exitCode=2
Jul 01 15:01:01.047756 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:01.047640    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerDied","Data":"9f4bcf7ece1a03d19ba8fabe93c556f808b96cd1d001aed334ad6ecb7fb8e4d7"}
Jul 01 15:01:01.049223 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:01.049201    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerStarted","Data":"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"}
Jul 01 15:01:01.049337 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:01.049247    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerStarted","Data":"4dd56c3a6d70f24f1f420b146fcbad4aaed8c4f1ecc4af143d01b24a58f6a6a5"}
Jul 01 15:01:01.978770 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:01.978738    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:02.055105 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:02.055071    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerStarted","Data":"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"}
Jul 01 15:01:02.055282 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:02.055215    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:02.075035 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:02.074981    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podStartSLOduration=3.074965078 podStartE2EDuration="3.074965078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:01:02.072455258 +0000 UTC m=+1310.788840016" watchObservedRunningTime="2026-07-01 15:01:02.074965078 +0000 UTC m=+1310.791349835"
Jul 01 15:01:02.180637 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:02.180597    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:03.061974 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.061811    2541 generic.go:356] "Generic (PLEG): container finished" podID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerID="61e0e894e88dbb09ed4459752392b9c06d63c0168232c43189a25afd5c66f442" exitCode=0
Jul 01 15:01:03.061974 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.061885    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerDied","Data":"61e0e894e88dbb09ed4459752392b9c06d63c0168232c43189a25afd5c66f442"}
Jul 01 15:01:03.062334 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.062271    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:03.063926 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.063893    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:03.203591 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.203566    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 15:01:03.239962 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.239920    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xmhd\" (UniqueName: \"kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd\") pod \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") "
Jul 01 15:01:03.240154 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.240001    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config\") pod \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") "
Jul 01 15:01:03.240154 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.240051    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") pod \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\" (UID: \"407379a1-739e-4c98-8bc7-8e2a23fb15e1\") "
Jul 01 15:01:03.240405 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.240373    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "407379a1-739e-4c98-8bc7-8e2a23fb15e1" (UID: "407379a1-739e-4c98-8bc7-8e2a23fb15e1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:01:03.242260 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.242228    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "407379a1-739e-4c98-8bc7-8e2a23fb15e1" (UID: "407379a1-739e-4c98-8bc7-8e2a23fb15e1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:01:03.242362 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.242232    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd" (OuterVolumeSpecName: "kube-api-access-9xmhd") pod "407379a1-739e-4c98-8bc7-8e2a23fb15e1" (UID: "407379a1-739e-4c98-8bc7-8e2a23fb15e1"). InnerVolumeSpecName "kube-api-access-9xmhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:01:03.340954 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.340837    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xmhd\" (UniqueName: \"kubernetes.io/projected/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-api-access-9xmhd\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:03.340954 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.340896    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/407379a1-739e-4c98-8bc7-8e2a23fb15e1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:03.340954 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:03.340907    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/407379a1-739e-4c98-8bc7-8e2a23fb15e1-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:04.068179 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.068099    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"
Jul 01 15:01:04.068179 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.068134    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5" event={"ID":"407379a1-739e-4c98-8bc7-8e2a23fb15e1","Type":"ContainerDied","Data":"af8b3ecd21df5b24f2823e158402147a379dceb2882526248d944cbb2f6af3c4"}
Jul 01 15:01:04.068670 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.068204    2541 scope.go:117] "RemoveContainer" containerID="9f4bcf7ece1a03d19ba8fabe93c556f808b96cd1d001aed334ad6ecb7fb8e4d7"
Jul 01 15:01:04.068670 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.068657    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:04.077435 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.077415    2541 scope.go:117] "RemoveContainer" containerID="61e0e894e88dbb09ed4459752392b9c06d63c0168232c43189a25afd5c66f442"
Jul 01 15:01:04.087161 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.087137    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 15:01:04.090765 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:04.090741    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0d08f-predictor-7fc6f58f4d-gx2v5"]
Jul 01 15:01:05.724481 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:05.724440    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" path="/var/lib/kubelet/pods/407379a1-739e-4c98-8bc7-8e2a23fb15e1/volumes"
Jul 01 15:01:09.073599 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:09.073572    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:09.074049 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:09.073992    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:09.993664 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:09.993630    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:10.683285 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:10.683252    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:19.075051 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:19.075016    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:20.087446 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.087407    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:20.135800 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.135761    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:01:20.136044 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.136002    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container" containerID="cri-o://c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be" gracePeriod=30
Jul 01 15:01:20.136134 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.136071    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kube-rbac-proxy" containerID="cri-o://655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf" gracePeriod=30
Jul 01 15:01:20.294767 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.294727    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:01:20.295199 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295181    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container"
Jul 01 15:01:20.295281 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295203    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container"
Jul 01 15:01:20.295281 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295241    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kube-rbac-proxy"
Jul 01 15:01:20.295281 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295251    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kube-rbac-proxy"
Jul 01 15:01:20.295435 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295351    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kserve-container"
Jul 01 15:01:20.295435 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.295367    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="407379a1-739e-4c98-8bc7-8e2a23fb15e1" containerName="kube-rbac-proxy"
Jul 01 15:01:20.297601 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.297580    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.299942 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.299922    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9eaf2-predictor-serving-cert\""
Jul 01 15:01:20.300069 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.300000    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9eaf2-kube-rbac-proxy-sar-config\""
Jul 01 15:01:20.307004 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.306982    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:01:20.390131 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.390010    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn9nh\" (UniqueName: \"kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.390287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.390139    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.390287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.390163    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.491424 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.491379    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn9nh\" (UniqueName: \"kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.491607 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.491471    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.491607 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.491497    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.492282 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.492261    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.494087 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.494069    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.499486 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.499460    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn9nh\" (UniqueName: \"kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh\") pod \"error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") " pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.610723 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.610680    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:20.751632 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:20.751590    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:01:20.753929 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:01:20.753903    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a428d0_0e4f_48f0_b889_ac1e37bd5ffa.slice/crio-145551b10d614025c7cfb84f076b361f5ec29f90a9a1c76b0e1c6f30f74546ec WatchSource:0}: Error finding container 145551b10d614025c7cfb84f076b361f5ec29f90a9a1c76b0e1c6f30f74546ec: Status 404 returned error can't find the container with id 145551b10d614025c7cfb84f076b361f5ec29f90a9a1c76b0e1c6f30f74546ec
Jul 01 15:01:21.155574 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.155488    2541 generic.go:356] "Generic (PLEG): container finished" podID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerID="655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf" exitCode=2
Jul 01 15:01:21.155574 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.155535    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerDied","Data":"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"}
Jul 01 15:01:21.157224 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.157200    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerStarted","Data":"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"}
Jul 01 15:01:21.157351 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.157254    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerStarted","Data":"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"}
Jul 01 15:01:21.157351 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.157284    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerStarted","Data":"145551b10d614025c7cfb84f076b361f5ec29f90a9a1c76b0e1c6f30f74546ec"}
Jul 01 15:01:21.157351 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.157316    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:21.157516 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.157449    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:21.158679 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.158659    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:01:21.176552 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.176508    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podStartSLOduration=1.176495087 podStartE2EDuration="1.176495087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:01:21.175066407 +0000 UTC m=+1329.891451164" watchObservedRunningTime="2026-07-01 15:01:21.176495087 +0000 UTC m=+1329.892879843"
Jul 01 15:01:21.752883 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:21.752819    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 15:01:22.162499 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:22.162465    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:01:23.687603 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.687577    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:01:23.820565 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.820479    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config\") pod \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") "
Jul 01 15:01:23.820565 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.820526    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls\") pod \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") "
Jul 01 15:01:23.820565 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.820556    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpvdz\" (UniqueName: \"kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz\") pod \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\" (UID: \"45fb3c58-5c38-4faa-99bf-844a1ab8a9de\") "
Jul 01 15:01:23.820892 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.820865    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "45fb3c58-5c38-4faa-99bf-844a1ab8a9de" (UID: "45fb3c58-5c38-4faa-99bf-844a1ab8a9de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:01:23.821025 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.821005    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:23.822856 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.822812    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz" (OuterVolumeSpecName: "kube-api-access-cpvdz") pod "45fb3c58-5c38-4faa-99bf-844a1ab8a9de" (UID: "45fb3c58-5c38-4faa-99bf-844a1ab8a9de"). InnerVolumeSpecName "kube-api-access-cpvdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:01:23.822996 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.822812    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "45fb3c58-5c38-4faa-99bf-844a1ab8a9de" (UID: "45fb3c58-5c38-4faa-99bf-844a1ab8a9de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:01:23.906412 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.906373    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:01:23.921923 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.921893    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:23.921923 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:23.921915    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpvdz\" (UniqueName: \"kubernetes.io/projected/45fb3c58-5c38-4faa-99bf-844a1ab8a9de-kube-api-access-cpvdz\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:01:24.172613 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.172585    2541 generic.go:356] "Generic (PLEG): container finished" podID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerID="c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be" exitCode=0
Jul 01 15:01:24.172777 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.172681    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerDied","Data":"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"}
Jul 01 15:01:24.172777 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.172695    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"
Jul 01 15:01:24.172777 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.172740    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg" event={"ID":"45fb3c58-5c38-4faa-99bf-844a1ab8a9de","Type":"ContainerDied","Data":"88cc49b99fe251491c274e721956ba020eadf7db0bac0ee6a535491dc41385af"}
Jul 01 15:01:24.172777 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.172766    2541 scope.go:117] "RemoveContainer" containerID="655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"
Jul 01 15:01:24.184059 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.184033    2541 scope.go:117] "RemoveContainer" containerID="c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"
Jul 01 15:01:24.192124 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.192096    2541 scope.go:117] "RemoveContainer" containerID="655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"
Jul 01 15:01:24.192431 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:01:24.192411    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf\": container with ID starting with 655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf not found: ID does not exist" containerID="655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"
Jul 01 15:01:24.192519 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.192443    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf"} err="failed to get container status \"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf\": rpc error: code = NotFound desc = could not find container \"655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf\": container with ID starting with 655f40ad8642b72bd68f8c4da6cee33e246124215bb4b9037d5f93bd61bffedf not found: ID does not exist"
Jul 01 15:01:24.192519 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.192466    2541 scope.go:117] "RemoveContainer" containerID="c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"
Jul 01 15:01:24.192737 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:01:24.192714    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be\": container with ID starting with c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be not found: ID does not exist" containerID="c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"
Jul 01 15:01:24.192793 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.192745    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be"} err="failed to get container status \"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be\": rpc error: code = NotFound desc = could not find container \"c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be\": container with ID starting with c2d92185b7b338c82eef3d9f3f81187adafbd04136d499c27a625bf7f3c018be not found: ID does not exist"
Jul 01 15:01:24.193787 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.193771    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:01:24.197136 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:24.197116    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7bec4-predictor-5fbd85475d-k2zzg"]
Jul 01 15:01:25.724751 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:25.724722    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" path="/var/lib/kubelet/pods/45fb3c58-5c38-4faa-99bf-844a1ab8a9de/volumes"
Jul 01 15:01:27.166655 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:27.166628    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:01:27.167180 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:27.167154    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:01:29.074074 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:29.074035    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:37.167194 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:37.167111    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:01:39.074446 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:39.074407    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 15:01:47.167475 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:47.167439    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:01:49.074856 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:49.074823    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:01:57.167130 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:01:57.167093    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:02:07.168415 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:07.168387    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:02:09.852431 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:09.852329    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:10.099155 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:10.099106    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:19.944990 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:19.944958    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.063772 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.063731    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.147561 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.147533    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:02:20.147808 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.147764    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container" containerID="cri-o://6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9" gracePeriod=30
Jul 01 15:02:20.147968 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.147815    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kube-rbac-proxy" containerID="cri-o://062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a" gracePeriod=30
Jul 01 15:02:20.155907 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.155882    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.167634 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.167607    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:02:20.168012 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.167998    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container"
Jul 01 15:02:20.168064 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.168013    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container"
Jul 01 15:02:20.168064 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.168034    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kube-rbac-proxy"
Jul 01 15:02:20.168064 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.168040    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kube-rbac-proxy"
Jul 01 15:02:20.168156 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.168107    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kserve-container"
Jul 01 15:02:20.168156 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.168115    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="45fb3c58-5c38-4faa-99bf-844a1ab8a9de" containerName="kube-rbac-proxy"
Jul 01 15:02:20.172653 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.172631    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.174913 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.174892    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6e167-predictor-serving-cert\""
Jul 01 15:02:20.174990 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.174904    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6e167-kube-rbac-proxy-sar-config\""
Jul 01 15:02:20.182715 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.182693    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:02:20.236504 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.236393    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.311294 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.311256    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.311470 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.311316    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.311470 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.311401    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqxzt\" (UniqueName: \"kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.334311 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.334279    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.412225 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.412186    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.412428 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.412241    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.412428 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.412301    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqxzt\" (UniqueName: \"kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.412428 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:02:20.412344    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-6e167-predictor-serving-cert: secret "error-404-isvc-6e167-predictor-serving-cert" not found
Jul 01 15:02:20.412428 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:02:20.412422    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls podName:cfc49d17-be73-45fa-af42-982df31e15cb nodeName:}" failed. No retries permitted until 2026-07-01 15:02:20.912402496 +0000 UTC m=+1389.628787244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls") pod "error-404-isvc-6e167-predictor-747c78f446-zrpn2" (UID: "cfc49d17-be73-45fa-af42-982df31e15cb") : secret "error-404-isvc-6e167-predictor-serving-cert" not found
Jul 01 15:02:20.412959 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.412937    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.420750 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.420724    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqxzt\" (UniqueName: \"kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.434754 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.434715    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.436003 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.435982    2541 generic.go:356] "Generic (PLEG): container finished" podID="0376aec0-5160-4084-8722-f11dd701e097" containerID="062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a" exitCode=2
Jul 01 15:02:20.436109 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.436047    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerDied","Data":"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"}
Jul 01 15:02:20.534530 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.534436    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:02:20.917635 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.917605    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:20.920053 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:20.920034    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") pod \"error-404-isvc-6e167-predictor-747c78f446-zrpn2\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") " pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:21.084071 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.084031    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:21.220503 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.220465    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:02:21.223396 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:02:21.223367    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc49d17_be73_45fa_af42_982df31e15cb.slice/crio-d4c5717d648f123f4cc4cfe2ce8cd3da47dd9ffcab9a89b9dc5baf53233d3e0b WatchSource:0}: Error finding container d4c5717d648f123f4cc4cfe2ce8cd3da47dd9ffcab9a89b9dc5baf53233d3e0b: Status 404 returned error can't find the container with id d4c5717d648f123f4cc4cfe2ce8cd3da47dd9ffcab9a89b9dc5baf53233d3e0b
Jul 01 15:02:21.443982 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.443949    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerStarted","Data":"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"}
Jul 01 15:02:21.444160 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.444006    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerStarted","Data":"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"}
Jul 01 15:02:21.444160 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.444035    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerStarted","Data":"d4c5717d648f123f4cc4cfe2ce8cd3da47dd9ffcab9a89b9dc5baf53233d3e0b"}
Jul 01 15:02:21.444160 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.444126    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:21.463233 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:21.463185    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podStartSLOduration=1.463169387 podStartE2EDuration="1.463169387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:02:21.462730835 +0000 UTC m=+1390.179115594" watchObservedRunningTime="2026-07-01 15:02:21.463169387 +0000 UTC m=+1390.179554144"
Jul 01 15:02:22.449326 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:22.449288    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:22.450426 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:22.450402    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:02:23.401281 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.401258    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:02:23.455637 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.455608    2541 generic.go:356] "Generic (PLEG): container finished" podID="0376aec0-5160-4084-8722-f11dd701e097" containerID="6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9" exitCode=0
Jul 01 15:02:23.456040 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.455654    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"
Jul 01 15:02:23.456040 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.455697    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerDied","Data":"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"}
Jul 01 15:02:23.456040 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.455758    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m" event={"ID":"0376aec0-5160-4084-8722-f11dd701e097","Type":"ContainerDied","Data":"4dd56c3a6d70f24f1f420b146fcbad4aaed8c4f1ecc4af143d01b24a58f6a6a5"}
Jul 01 15:02:23.456040 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.455790    2541 scope.go:117] "RemoveContainer" containerID="062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"
Jul 01 15:02:23.456234 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.456203    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:02:23.464291 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.464274    2541 scope.go:117] "RemoveContainer" containerID="6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"
Jul 01 15:02:23.471747 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.471727    2541 scope.go:117] "RemoveContainer" containerID="062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"
Jul 01 15:02:23.472029 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:02:23.472008    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a\": container with ID starting with 062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a not found: ID does not exist" containerID="062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"
Jul 01 15:02:23.472121 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.472040    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a"} err="failed to get container status \"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a\": rpc error: code = NotFound desc = could not find container \"062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a\": container with ID starting with 062a496e652dc42a71516b1f18625b2fe50490e897508b39c10f1c690a61ff5a not found: ID does not exist"
Jul 01 15:02:23.472121 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.472063    2541 scope.go:117] "RemoveContainer" containerID="6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"
Jul 01 15:02:23.472325 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:02:23.472307    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9\": container with ID starting with 6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9 not found: ID does not exist" containerID="6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"
Jul 01 15:02:23.472377 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.472332    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9"} err="failed to get container status \"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9\": rpc error: code = NotFound desc = could not find container \"6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9\": container with ID starting with 6c620bcf077aad41f6a7b5fcc031dc9d5d80d53c3d214948f0c6c3f207a0bab9 not found: ID does not exist"
Jul 01 15:02:23.540200 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.540094    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config\") pod \"0376aec0-5160-4084-8722-f11dd701e097\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") "
Jul 01 15:02:23.540351 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.540205    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") pod \"0376aec0-5160-4084-8722-f11dd701e097\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") "
Jul 01 15:02:23.540500 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.540481    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsdct\" (UniqueName: \"kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct\") pod \"0376aec0-5160-4084-8722-f11dd701e097\" (UID: \"0376aec0-5160-4084-8722-f11dd701e097\") "
Jul 01 15:02:23.541218 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.540991    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0376aec0-5160-4084-8722-f11dd701e097" (UID: "0376aec0-5160-4084-8722-f11dd701e097"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:02:23.543340 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.543314    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0376aec0-5160-4084-8722-f11dd701e097" (UID: "0376aec0-5160-4084-8722-f11dd701e097"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:02:23.543434 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.543321    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct" (OuterVolumeSpecName: "kube-api-access-fsdct") pod "0376aec0-5160-4084-8722-f11dd701e097" (UID: "0376aec0-5160-4084-8722-f11dd701e097"). InnerVolumeSpecName "kube-api-access-fsdct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:02:23.642067 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.642033    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0376aec0-5160-4084-8722-f11dd701e097-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:02:23.642067 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.642061    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0376aec0-5160-4084-8722-f11dd701e097-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:02:23.642067 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.642074    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsdct\" (UniqueName: \"kubernetes.io/projected/0376aec0-5160-4084-8722-f11dd701e097-kube-api-access-fsdct\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:02:23.772636 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.772608    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:02:23.774570 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:23.774542    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-102b8-predictor-548dfbff4-mm86m"]
Jul 01 15:02:25.725660 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:25.725628    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0376aec0-5160-4084-8722-f11dd701e097" path="/var/lib/kubelet/pods/0376aec0-5160-4084-8722-f11dd701e097/volumes"
Jul 01 15:02:28.461329 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:28.461302    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:02:28.461795 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:28.461772    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:02:38.462613 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:38.462576    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:02:48.462398 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:48.462359    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:02:58.462172 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:02:58.462133    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 01 15:03:08.462799 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:03:08.462715    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:03:30.177064 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:03:30.177029    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:03:30.584958 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:03:30.584874    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:04:11.873617 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:11.873587    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:04:11.878664 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:11.878639    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:04:11.879750 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:11.879730    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:04:11.885131 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:11.885109    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:04:22.331220 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:22.331185    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:04:54.538548 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:04:54.538515    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:05:19.558678 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:05:19.558643    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:09:11.907196 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:09:11.907166    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:09:11.911996 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:09:11.911977    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:09:11.912966 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:09:11.912948    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:09:11.917876 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:09:11.917832    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:10:06.699900 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:06.699865    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.013005 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.012925    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.087173 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.087137    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.171076 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.171048    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.183686 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.183660    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:10:35.184095 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184082    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container"
Jul 01 15:10:35.184095 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184096    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container"
Jul 01 15:10:35.184186 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184108    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kube-rbac-proxy"
Jul 01 15:10:35.184186 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184113    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kube-rbac-proxy"
Jul 01 15:10:35.184186 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184183    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kube-rbac-proxy"
Jul 01 15:10:35.184284 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.184193    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="0376aec0-5160-4084-8722-f11dd701e097" containerName="kserve-container"
Jul 01 15:10:35.187097 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.187081    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.189435 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.189411    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6f42a-predictor-serving-cert\""
Jul 01 15:10:35.189549 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.189507    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6f42a-kube-rbac-proxy-sar-config\""
Jul 01 15:10:35.198180 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.198154    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:10:35.228631 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.228593    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:10:35.228908 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.228863    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" containerID="cri-o://7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11" gracePeriod=30
Jul 01 15:10:35.229032 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.228910    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kube-rbac-proxy" containerID="cri-o://84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111" gracePeriod=30
Jul 01 15:10:35.234287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.234004    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hcl6\" (UniqueName: \"kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.234287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.234088    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.234287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.234167    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.249982 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.249950    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.334651 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.334571    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.334651 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.334630    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.334909 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.334660    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hcl6\" (UniqueName: \"kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.334909 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:10:35.334716    2541 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-serving-cert: secret "error-404-isvc-6f42a-predictor-serving-cert" not found
Jul 01 15:10:35.334909 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:10:35.334786    2541 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls podName:398874b0-8fd2-4d31-b7b1-3e07e069ce1a nodeName:}" failed. No retries permitted until 2026-07-01 15:10:35.83476612 +0000 UTC m=+1884.551150879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls") pod "error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" (UID: "398874b0-8fd2-4d31-b7b1-3e07e069ce1a") : secret "error-404-isvc-6f42a-predictor-serving-cert" not found
Jul 01 15:10:35.335713 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.335689    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.346123 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.346100    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hcl6\" (UniqueName: \"kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.435477 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.435441    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:35.756481 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.756447    2541 generic.go:356] "Generic (PLEG): container finished" podID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerID="84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111" exitCode=2
Jul 01 15:10:35.756669 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.756528    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerDied","Data":"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"}
Jul 01 15:10:35.838215 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.838182    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.840950 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.840925    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") pod \"error-404-isvc-6f42a-predictor-5b8658b774-p5rwr\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") " pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:35.880038 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:35.880009    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:10:36.097762 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.097671    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:36.237918 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.237893    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:10:36.240924 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:10:36.240888    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398874b0_8fd2_4d31_b7b1_3e07e069ce1a.slice/crio-308ce551d68e7a0388013c96974455bd7b79ddd9a45a9fd68d555320101f2ee2 WatchSource:0}: Error finding container 308ce551d68e7a0388013c96974455bd7b79ddd9a45a9fd68d555320101f2ee2: Status 404 returned error can't find the container with id 308ce551d68e7a0388013c96974455bd7b79ddd9a45a9fd68d555320101f2ee2
Jul 01 15:10:36.763033 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.762998    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerStarted","Data":"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"}
Jul 01 15:10:36.763272 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.763065    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerStarted","Data":"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"}
Jul 01 15:10:36.763272 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.763093    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerStarted","Data":"308ce551d68e7a0388013c96974455bd7b79ddd9a45a9fd68d555320101f2ee2"}
Jul 01 15:10:36.763272 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.763169    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:36.782828 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:36.782783    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podStartSLOduration=1.782768419 podStartE2EDuration="1.782768419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:10:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:10:36.780266538 +0000 UTC m=+1885.496651330" watchObservedRunningTime="2026-07-01 15:10:36.782768419 +0000 UTC m=+1885.499153175"
Jul 01 15:10:37.163493 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:37.163451    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 01 15:10:37.167761 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:37.167735    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 01 15:10:37.768403 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:37.768369    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:37.769450 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:37.769422    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:10:38.482853 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.482815    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:10:38.562107 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.562033    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls\") pod \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") "
Jul 01 15:10:38.562107 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.562073    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn9nh\" (UniqueName: \"kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh\") pod \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") "
Jul 01 15:10:38.562335 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.562114    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config\") pod \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\" (UID: \"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa\") "
Jul 01 15:10:38.562540 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.562515    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" (UID: "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:10:38.564148 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.564117    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" (UID: "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:10:38.564349 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.564331    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh" (OuterVolumeSpecName: "kube-api-access-dn9nh") pod "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" (UID: "15a428d0-0e4f-48f0-b889-ac1e37bd5ffa"). InnerVolumeSpecName "kube-api-access-dn9nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:10:38.663270 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.663224    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:10:38.663270 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.663259    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn9nh\" (UniqueName: \"kubernetes.io/projected/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-api-access-dn9nh\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:10:38.663270 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.663270    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:10:38.774368 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774337    2541 generic.go:356] "Generic (PLEG): container finished" podID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerID="7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11" exitCode=0
Jul 01 15:10:38.774538 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774393    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"
Jul 01 15:10:38.774538 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774431    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerDied","Data":"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"}
Jul 01 15:10:38.774538 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774498    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj" event={"ID":"15a428d0-0e4f-48f0-b889-ac1e37bd5ffa","Type":"ContainerDied","Data":"145551b10d614025c7cfb84f076b361f5ec29f90a9a1c76b0e1c6f30f74546ec"}
Jul 01 15:10:38.774538 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774525    2541 scope.go:117] "RemoveContainer" containerID="84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"
Jul 01 15:10:38.775018 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.774987    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:10:38.784003 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.783984    2541 scope.go:117] "RemoveContainer" containerID="7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"
Jul 01 15:10:38.792170 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.792148    2541 scope.go:117] "RemoveContainer" containerID="84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"
Jul 01 15:10:38.792436 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:10:38.792416    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111\": container with ID starting with 84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111 not found: ID does not exist" containerID="84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"
Jul 01 15:10:38.792476 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.792447    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111"} err="failed to get container status \"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111\": rpc error: code = NotFound desc = could not find container \"84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111\": container with ID starting with 84101839277c369ab18ce6ee3f7c240c202c76d3ec8ce8016af025d150560111 not found: ID does not exist"
Jul 01 15:10:38.792476 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.792465    2541 scope.go:117] "RemoveContainer" containerID="7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"
Jul 01 15:10:38.792699 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:10:38.792680    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11\": container with ID starting with 7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11 not found: ID does not exist" containerID="7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"
Jul 01 15:10:38.792738 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.792707    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11"} err="failed to get container status \"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11\": rpc error: code = NotFound desc = could not find container \"7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11\": container with ID starting with 7eb15415b22a616eab89c182826ad71224afcbf7e1ac1c47336eca3208f1ff11 not found: ID does not exist"
Jul 01 15:10:38.795602 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.795579    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:10:38.801887 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:38.801867    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9eaf2-predictor-86d6f679c4-lg4jj"]
Jul 01 15:10:39.725290 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:39.725256    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" path="/var/lib/kubelet/pods/15a428d0-0e4f-48f0-b889-ac1e37bd5ffa/volumes"
Jul 01 15:10:43.780295 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:43.780266    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:10:43.780695 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:43.780662    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:10:53.781025 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:10:53.780985    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:11:03.780908 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:03.780858    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:11:13.781573 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:13.781531    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 01 15:11:23.780995 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:23.780963    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:11:35.166975 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:35.166942    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:35.524195 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:35.524108    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:44.986376 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:44.986338    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:11:44.986833 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:44.986634    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container" containerID="cri-o://19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97" gracePeriod=30
Jul 01 15:11:44.986833 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:44.986668    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kube-rbac-proxy" containerID="cri-o://5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73" gracePeriod=30
Jul 01 15:11:45.183199 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183165    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:11:45.183541 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183529    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kube-rbac-proxy"
Jul 01 15:11:45.183586 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183543    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kube-rbac-proxy"
Jul 01 15:11:45.183586 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183556    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container"
Jul 01 15:11:45.183586 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183562    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container"
Jul 01 15:11:45.183684 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183638    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kube-rbac-proxy"
Jul 01 15:11:45.183684 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.183649    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a428d0-0e4f-48f0-b889-ac1e37bd5ffa" containerName="kserve-container"
Jul 01 15:11:45.186700 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.186681    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.189360 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.189338    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f5851-predictor-serving-cert\""
Jul 01 15:11:45.189492 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.189372    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f5851-kube-rbac-proxy-sar-config\""
Jul 01 15:11:45.204359 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.204326    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:11:45.259431 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.259394    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.346566 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.346527    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l48h\" (UniqueName: \"kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.346566 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.346569    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.346795 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.346601    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.351575 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.351541    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.425731 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.425699    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.433296 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.433210    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:11:45.433490 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.433465    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container" containerID="cri-o://913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6" gracePeriod=30
Jul 01 15:11:45.433586 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.433506    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kube-rbac-proxy" containerID="cri-o://d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd" gracePeriod=30
Jul 01 15:11:45.447896 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.447867    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l48h\" (UniqueName: \"kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.448012 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.447910    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.448012 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.447940    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.448540 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.448518    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.450401 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.450377    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.456700 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.456672    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l48h\" (UniqueName: \"kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h\") pod \"error-404-isvc-f5851-predictor-95c7775db-wcjpb\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") " pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.491826 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.491791    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.496775 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.496755    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:45.581881 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.581830    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.647350 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.647317    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:11:45.650420 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:11:45.650387    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e1c3d5_96d6_4d20_a44d_94df98d50628.slice/crio-eab58021635bd4e207307c6543b0ce66e39dbba34e5099f19cdd282732dfaaec WatchSource:0}: Error finding container eab58021635bd4e207307c6543b0ce66e39dbba34e5099f19cdd282732dfaaec: Status 404 returned error can't find the container with id eab58021635bd4e207307c6543b0ce66e39dbba34e5099f19cdd282732dfaaec
Jul 01 15:11:45.698600 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.698575    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:45.700005 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.699978    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:11:45.703408 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.703392    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.706540 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.706521    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-23f07-predictor-serving-cert\""
Jul 01 15:11:45.706662 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.706564    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-23f07-kube-rbac-proxy-sar-config\""
Jul 01 15:11:45.717062 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.717042    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:11:45.850504 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.850472    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.850713 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.850691    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkv2l\" (UniqueName: \"kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.850831 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.850733    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.952051 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.951962    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.952217 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.952083    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkv2l\" (UniqueName: \"kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.952217 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.952111    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.952737 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.952717    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.954758 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.954735    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.961683 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.961663    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkv2l\" (UniqueName: \"kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l\") pod \"error-404-isvc-23f07-predictor-5c87794f48-svfqr\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") " pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:45.994135 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:45.994105    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:46.015124 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.015087    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:46.092478 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.092440    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerStarted","Data":"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"}
Jul 01 15:11:46.092652 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.092521    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerStarted","Data":"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"}
Jul 01 15:11:46.092652 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.092563    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:46.092652 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.092591    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerStarted","Data":"eab58021635bd4e207307c6543b0ce66e39dbba34e5099f19cdd282732dfaaec"}
Jul 01 15:11:46.094453 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.094425    2541 generic.go:356] "Generic (PLEG): container finished" podID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerID="d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd" exitCode=2
Jul 01 15:11:46.094583 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.094508    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerDied","Data":"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"}
Jul 01 15:11:46.096526 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.096504    2541 generic.go:356] "Generic (PLEG): container finished" podID="cfc49d17-be73-45fa-af42-982df31e15cb" containerID="5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73" exitCode=2
Jul 01 15:11:46.096637 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.096547    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerDied","Data":"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"}
Jul 01 15:11:46.146047 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.145962    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podStartSLOduration=1.14594368 podStartE2EDuration="1.14594368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:11:46.144445928 +0000 UTC m=+1954.860830697" watchObservedRunningTime="2026-07-01 15:11:46.14594368 +0000 UTC m=+1954.862328436"
Jul 01 15:11:46.171661 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.171635    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:11:46.173545 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:11:46.173523    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac82ba03_c329_4df9_9e6e_fa75a096fa4b.slice/crio-d8653304b67edeaaadc110a069733b566559b726ff92cc6d81aa06be24e96380 WatchSource:0}: Error finding container d8653304b67edeaaadc110a069733b566559b726ff92cc6d81aa06be24e96380: Status 404 returned error can't find the container with id d8653304b67edeaaadc110a069733b566559b726ff92cc6d81aa06be24e96380
Jul 01 15:11:46.226269 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:46.226245    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:11:47.102813 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.102776    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerStarted","Data":"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"}
Jul 01 15:11:47.103314 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.102877    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:47.103314 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.102913    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerStarted","Data":"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"}
Jul 01 15:11:47.103314 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.102945    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerStarted","Data":"d8653304b67edeaaadc110a069733b566559b726ff92cc6d81aa06be24e96380"}
Jul 01 15:11:47.103314 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.103126    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:47.103314 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.103175    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:47.104712 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.104683    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:11:47.104887 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.104683    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:11:47.121510 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:47.121464    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podStartSLOduration=2.121449949 podStartE2EDuration="2.121449949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:11:47.12058591 +0000 UTC m=+1955.836970668" watchObservedRunningTime="2026-07-01 15:11:47.121449949 +0000 UTC m=+1955.837834706"
Jul 01 15:11:48.107262 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.107227    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:11:48.107658 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.107327    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:11:48.439996 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.439960    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:11:48.577163 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.577114    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") pod \"cfc49d17-be73-45fa-af42-982df31e15cb\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") "
Jul 01 15:11:48.577336 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.577173    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config\") pod \"cfc49d17-be73-45fa-af42-982df31e15cb\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") "
Jul 01 15:11:48.577336 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.577237    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqxzt\" (UniqueName: \"kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt\") pod \"cfc49d17-be73-45fa-af42-982df31e15cb\" (UID: \"cfc49d17-be73-45fa-af42-982df31e15cb\") "
Jul 01 15:11:48.577581 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.577546    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cfc49d17-be73-45fa-af42-982df31e15cb" (UID: "cfc49d17-be73-45fa-af42-982df31e15cb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:11:48.579466 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.579429    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt" (OuterVolumeSpecName: "kube-api-access-lqxzt") pod "cfc49d17-be73-45fa-af42-982df31e15cb" (UID: "cfc49d17-be73-45fa-af42-982df31e15cb"). InnerVolumeSpecName "kube-api-access-lqxzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:11:48.579576 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.579513    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cfc49d17-be73-45fa-af42-982df31e15cb" (UID: "cfc49d17-be73-45fa-af42-982df31e15cb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:11:48.678919 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.678817    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cfc49d17-be73-45fa-af42-982df31e15cb-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:48.678919 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.678872    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cfc49d17-be73-45fa-af42-982df31e15cb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:48.678919 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.678887    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqxzt\" (UniqueName: \"kubernetes.io/projected/cfc49d17-be73-45fa-af42-982df31e15cb-kube-api-access-lqxzt\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:48.776092 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.776052    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.41:8643/healthz\": dial tcp 10.132.0.41:8643: connect: connection refused"
Jul 01 15:11:48.877134 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.877109    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:11:48.981501 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.981409    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config\") pod \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") "
Jul 01 15:11:48.981501 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.981459    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") pod \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") "
Jul 01 15:11:48.981741 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.981512    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hcl6\" (UniqueName: \"kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6\") pod \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\" (UID: \"398874b0-8fd2-4d31-b7b1-3e07e069ce1a\") "
Jul 01 15:11:48.981806 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.981755    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "398874b0-8fd2-4d31-b7b1-3e07e069ce1a" (UID: "398874b0-8fd2-4d31-b7b1-3e07e069ce1a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:11:48.983712 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.983677    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "398874b0-8fd2-4d31-b7b1-3e07e069ce1a" (UID: "398874b0-8fd2-4d31-b7b1-3e07e069ce1a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:11:48.983805 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:48.983711    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6" (OuterVolumeSpecName: "kube-api-access-2hcl6") pod "398874b0-8fd2-4d31-b7b1-3e07e069ce1a" (UID: "398874b0-8fd2-4d31-b7b1-3e07e069ce1a"). InnerVolumeSpecName "kube-api-access-2hcl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:11:49.082911 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.082871    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.082911 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.082905    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.082911 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.082915    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hcl6\" (UniqueName: \"kubernetes.io/projected/398874b0-8fd2-4d31-b7b1-3e07e069ce1a-kube-api-access-2hcl6\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.113716 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.113687    2541 generic.go:356] "Generic (PLEG): container finished" podID="cfc49d17-be73-45fa-af42-982df31e15cb" containerID="19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97" exitCode=0
Jul 01 15:11:49.114153 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.113730    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"
Jul 01 15:11:49.114153 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.113762    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerDied","Data":"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"}
Jul 01 15:11:49.114153 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.113817    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2" event={"ID":"cfc49d17-be73-45fa-af42-982df31e15cb","Type":"ContainerDied","Data":"d4c5717d648f123f4cc4cfe2ce8cd3da47dd9ffcab9a89b9dc5baf53233d3e0b"}
Jul 01 15:11:49.114153 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.113878    2541 scope.go:117] "RemoveContainer" containerID="5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"
Jul 01 15:11:49.116023 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.115997    2541 generic.go:356] "Generic (PLEG): container finished" podID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerID="913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6" exitCode=0
Jul 01 15:11:49.116108 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.116036    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerDied","Data":"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"}
Jul 01 15:11:49.116108 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.116058    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"
Jul 01 15:11:49.116108 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.116087    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr" event={"ID":"398874b0-8fd2-4d31-b7b1-3e07e069ce1a","Type":"ContainerDied","Data":"308ce551d68e7a0388013c96974455bd7b79ddd9a45a9fd68d555320101f2ee2"}
Jul 01 15:11:49.124286 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.124269    2541 scope.go:117] "RemoveContainer" containerID="19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"
Jul 01 15:11:49.132335 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.132320    2541 scope.go:117] "RemoveContainer" containerID="5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"
Jul 01 15:11:49.132581 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:11:49.132562    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73\": container with ID starting with 5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73 not found: ID does not exist" containerID="5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"
Jul 01 15:11:49.132617 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.132590    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73"} err="failed to get container status \"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73\": rpc error: code = NotFound desc = could not find container \"5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73\": container with ID starting with 5fb1f50d9234ec9ee1d6fc9caebf0b613391aaa813d747ff9d6ae1b4d5c77f73 not found: ID does not exist"
Jul 01 15:11:49.132617 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.132609    2541 scope.go:117] "RemoveContainer" containerID="19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"
Jul 01 15:11:49.132863 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:11:49.132833    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97\": container with ID starting with 19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97 not found: ID does not exist" containerID="19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"
Jul 01 15:11:49.132923 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.132867    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97"} err="failed to get container status \"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97\": rpc error: code = NotFound desc = could not find container \"19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97\": container with ID starting with 19ec5292c9eda7d3a677ce8d30ceb8836d1c01b0ca8d9ad6030ae89634266d97 not found: ID does not exist"
Jul 01 15:11:49.132923 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.132881    2541 scope.go:117] "RemoveContainer" containerID="d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"
Jul 01 15:11:49.137403 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.137374    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:11:49.140218 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.140190    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e167-predictor-747c78f446-zrpn2"]
Jul 01 15:11:49.141329 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.141301    2541 scope.go:117] "RemoveContainer" containerID="913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"
Jul 01 15:11:49.149431 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.149413    2541 scope.go:117] "RemoveContainer" containerID="d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"
Jul 01 15:11:49.149728 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:11:49.149708    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd\": container with ID starting with d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd not found: ID does not exist" containerID="d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"
Jul 01 15:11:49.149776 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.149730    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:11:49.149776 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.149736    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd"} err="failed to get container status \"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd\": rpc error: code = NotFound desc = could not find container \"d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd\": container with ID starting with d1bd45e664d85b30e53a579f4a0088a91857f4369ba2deca74d9289e36448ecd not found: ID does not exist"
Jul 01 15:11:49.149776 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.149754    2541 scope.go:117] "RemoveContainer" containerID="913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"
Jul 01 15:11:49.150024 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:11:49.150007    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6\": container with ID starting with 913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6 not found: ID does not exist" containerID="913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"
Jul 01 15:11:49.150071 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.150030    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6"} err="failed to get container status \"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6\": rpc error: code = NotFound desc = could not find container \"913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6\": container with ID starting with 913196d15b7ffbf9c62a2fc582365b32a88f8e3d4f6bb9838ad1ec04152e12a6 not found: ID does not exist"
Jul 01 15:11:49.153308 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.153289    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6f42a-predictor-5b8658b774-p5rwr"]
Jul 01 15:11:49.724759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.724728    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" path="/var/lib/kubelet/pods/398874b0-8fd2-4d31-b7b1-3e07e069ce1a/volumes"
Jul 01 15:11:49.725239 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:49.725225    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" path="/var/lib/kubelet/pods/cfc49d17-be73-45fa-af42-982df31e15cb/volumes"
Jul 01 15:11:53.111738 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:53.111707    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:11:53.112239 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:53.112223    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:11:53.112303 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:53.112285    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:11:53.112629 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:11:53.112607    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:12:03.112456 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:03.112372    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:12:03.112833 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:03.112671    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:12:13.113071 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:13.113033    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:12:13.113500 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:13.113037    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:12:23.113030 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:23.112989    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:12:23.113472 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:23.112992    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 15:12:33.113016 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:33.112986    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:12:33.113472 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:33.113272    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:12:55.142009 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:55.141958    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:12:55.270543 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:55.270505    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:12:55.502134 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:55.502103    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:12:55.840073 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:12:55.839967    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:13:14.784280 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:13:14.784222    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:13:14.866708 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:13:14.866674    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:14:11.936766 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:14:11.936739    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:14:11.941551 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:14:11.941529    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:14:11.945162 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:14:11.945143    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:14:11.949799 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:14:11.949781    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:15:16.137340 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:15:16.137264    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:16:48.550276 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:16:48.550174    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:18:48.773535 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:18:48.773498    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:19:11.969487 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:19:11.969457    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:19:11.974215 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:19:11.974189    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:19:11.977261 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:19:11.977233    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:19:11.981612 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:19:11.981597    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:21:10.287863 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.287760    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:21:10.288397 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.288030    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" containerID="cri-o://417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a" gracePeriod=30
Jul 01 15:21:10.288397 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.288073    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kube-rbac-proxy" containerID="cri-o://305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6" gracePeriod=30
Jul 01 15:21:10.655439 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.655354    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:21:10.735635 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.735599    2541 generic.go:356] "Generic (PLEG): container finished" podID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerID="305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6" exitCode=2
Jul 01 15:21:10.735833 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:10.735673    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerDied","Data":"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"}
Jul 01 15:21:13.108264 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.108210    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.43:8643/healthz\": dial tcp 10.132.0.43:8643: connect: connection refused"
Jul 01 15:21:13.112786 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.112752    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 01 15:21:13.438097 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.438072    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:21:13.573382 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.573336    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config\") pod \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") "
Jul 01 15:21:13.573530 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.573428    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkv2l\" (UniqueName: \"kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l\") pod \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") "
Jul 01 15:21:13.573530 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.573502    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls\") pod \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\" (UID: \"ac82ba03-c329-4df9-9e6e-fa75a096fa4b\") "
Jul 01 15:21:13.573757 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.573730    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ac82ba03-c329-4df9-9e6e-fa75a096fa4b" (UID: "ac82ba03-c329-4df9-9e6e-fa75a096fa4b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:21:13.575608 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.575581    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l" (OuterVolumeSpecName: "kube-api-access-dkv2l") pod "ac82ba03-c329-4df9-9e6e-fa75a096fa4b" (UID: "ac82ba03-c329-4df9-9e6e-fa75a096fa4b"). InnerVolumeSpecName "kube-api-access-dkv2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:21:13.575608 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.575587    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ac82ba03-c329-4df9-9e6e-fa75a096fa4b" (UID: "ac82ba03-c329-4df9-9e6e-fa75a096fa4b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:21:13.674548 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.674471    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkv2l\" (UniqueName: \"kubernetes.io/projected/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-api-access-dkv2l\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.674548 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.674498    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.674548 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.674508    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac82ba03-c329-4df9-9e6e-fa75a096fa4b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.751481 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.751455    2541 generic.go:356] "Generic (PLEG): container finished" podID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerID="417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a" exitCode=0
Jul 01 15:21:13.751631 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.751502    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"
Jul 01 15:21:13.751631 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.751539    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerDied","Data":"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"}
Jul 01 15:21:13.751631 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.751602    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr" event={"ID":"ac82ba03-c329-4df9-9e6e-fa75a096fa4b","Type":"ContainerDied","Data":"d8653304b67edeaaadc110a069733b566559b726ff92cc6d81aa06be24e96380"}
Jul 01 15:21:13.751756 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.751641    2541 scope.go:117] "RemoveContainer" containerID="305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"
Jul 01 15:21:13.760817 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.760799    2541 scope.go:117] "RemoveContainer" containerID="417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"
Jul 01 15:21:13.768249 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.768231    2541 scope.go:117] "RemoveContainer" containerID="305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"
Jul 01 15:21:13.768590 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:21:13.768558    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6\": container with ID starting with 305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6 not found: ID does not exist" containerID="305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"
Jul 01 15:21:13.768693 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.768590    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6"} err="failed to get container status \"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6\": rpc error: code = NotFound desc = could not find container \"305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6\": container with ID starting with 305bd57c349ff725e6c4dd219bc62aa5cd7060d26de95935248e85b77f27e8b6 not found: ID does not exist"
Jul 01 15:21:13.768693 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.768608    2541 scope.go:117] "RemoveContainer" containerID="417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"
Jul 01 15:21:13.768870 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:21:13.768826    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a\": container with ID starting with 417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a not found: ID does not exist" containerID="417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"
Jul 01 15:21:13.768935 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.768880    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a"} err="failed to get container status \"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a\": rpc error: code = NotFound desc = could not find container \"417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a\": container with ID starting with 417fdb250911950c14a17e1b7846f628db9e5b964cf7616e1a6ad056367a427a not found: ID does not exist"
Jul 01 15:21:13.773019 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.772991    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:21:13.775440 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:13.775422    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23f07-predictor-5c87794f48-svfqr"]
Jul 01 15:21:15.175346 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:15.175312    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:21:15.725136 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:21:15.725105    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" path="/var/lib/kubelet/pods/ac82ba03-c329-4df9-9e6e-fa75a096fa4b/volumes"
Jul 01 15:24:05.273384 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:24:05.273291    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:24:11.999644 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:24:11.999615    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:24:12.004327 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:24:12.004305    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:24:12.007139 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:24:12.007113    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:24:12.011758 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:24:12.011737    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:28:04.532541 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:28:04.532501    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:12.029749 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:12.029720    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:29:12.034898 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:12.034874    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:29:12.038942 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:12.038920    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:29:12.043597 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:12.043578    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:29:14.597801 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.597772    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:14.787084 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.787051    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:14.850959 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.850872    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:29:14.851110 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.851083    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container" containerID="cri-o://2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28" gracePeriod=30
Jul 01 15:29:14.851185 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.851141    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kube-rbac-proxy" containerID="cri-o://65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168" gracePeriod=30
Jul 01 15:29:14.942373 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.942337    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:14.998340 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.998307    2541 generic.go:356] "Generic (PLEG): container finished" podID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerID="65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168" exitCode=2
Jul 01 15:29:14.998506 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:14.998370    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerDied","Data":"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"}
Jul 01 15:29:16.193047 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193015    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4cr6f/must-gather-gsz6d"]
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193362    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193372    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193397    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193403    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193412    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193418    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193426    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193433    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193445    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193444 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193451    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193460    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193465    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193533    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193544    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kserve-container"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193550    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc49d17-be73-45fa-af42-982df31e15cb" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193557    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kube-rbac-proxy"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193564    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="398874b0-8fd2-4d31-b7b1-3e07e069ce1a" containerName="kserve-container"
Jul 01 15:29:16.193739 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.193569    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac82ba03-c329-4df9-9e6e-fa75a096fa4b" containerName="kserve-container"
Jul 01 15:29:16.196542 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.196526    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.199400 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.199376    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4cr6f\"/\"default-dockercfg-g7b8w\""
Jul 01 15:29:16.200178 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.200161    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cr6f\"/\"kube-root-ca.crt\""
Jul 01 15:29:16.202864 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.202823    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cr6f\"/\"openshift-service-ca.crt\""
Jul 01 15:29:16.203133 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.203104    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:16.210126 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.210103    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cr6f/must-gather-gsz6d"]
Jul 01 15:29:16.235583 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.235549    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.235735 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.235619    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2k4\" (UniqueName: \"kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.284288 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.284257    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:16.336101 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.336071    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2k4\" (UniqueName: \"kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.336238 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.336143    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.336510 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.336486    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.346930 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.346895    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2k4\" (UniqueName: \"kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4\") pod \"must-gather-gsz6d\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") " pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.519341 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.519223    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:16.659135 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.659111    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cr6f/must-gather-gsz6d"]
Jul 01 15:29:16.660861 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:29:16.660822    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c00e6ce_bdaa_4938_a53a_081c7fd577c5.slice/crio-c292d4cf5cdffbf15c0dec1fa829e0083d13e90a26c6d266e9b04d19a23a1b31 WatchSource:0}: Error finding container c292d4cf5cdffbf15c0dec1fa829e0083d13e90a26c6d266e9b04d19a23a1b31: Status 404 returned error can't find the container with id c292d4cf5cdffbf15c0dec1fa829e0083d13e90a26c6d266e9b04d19a23a1b31
Jul 01 15:29:16.663139 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:16.663124    2541 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 15:29:17.008498 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:17.008464    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" event={"ID":"4c00e6ce-bdaa-4938-a53a-081c7fd577c5","Type":"ContainerStarted","Data":"c292d4cf5cdffbf15c0dec1fa829e0083d13e90a26c6d266e9b04d19a23a1b31"}
Jul 01 15:29:18.107819 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.107775    2541 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused"
Jul 01 15:29:18.279430 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.279390    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:29:18.354792 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.354647    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls\") pod \"35e1c3d5-96d6-4d20-a44d-94df98d50628\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") "
Jul 01 15:29:18.354792 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.354779    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l48h\" (UniqueName: \"kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h\") pod \"35e1c3d5-96d6-4d20-a44d-94df98d50628\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") "
Jul 01 15:29:18.355188 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.354831    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config\") pod \"35e1c3d5-96d6-4d20-a44d-94df98d50628\" (UID: \"35e1c3d5-96d6-4d20-a44d-94df98d50628\") "
Jul 01 15:29:18.355303 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.355199    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "35e1c3d5-96d6-4d20-a44d-94df98d50628" (UID: "35e1c3d5-96d6-4d20-a44d-94df98d50628"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:29:18.357428 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.357373    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "35e1c3d5-96d6-4d20-a44d-94df98d50628" (UID: "35e1c3d5-96d6-4d20-a44d-94df98d50628"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:29:18.357595 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.357570    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h" (OuterVolumeSpecName: "kube-api-access-5l48h") pod "35e1c3d5-96d6-4d20-a44d-94df98d50628" (UID: "35e1c3d5-96d6-4d20-a44d-94df98d50628"). InnerVolumeSpecName "kube-api-access-5l48h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:29:18.455607 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.455564    2541 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/35e1c3d5-96d6-4d20-a44d-94df98d50628-proxy-tls\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:29:18.455607 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.455603    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l48h\" (UniqueName: \"kubernetes.io/projected/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-api-access-5l48h\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:29:18.455607 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:18.455614    2541 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/35e1c3d5-96d6-4d20-a44d-94df98d50628-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:29:19.023385 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.023343    2541 generic.go:356] "Generic (PLEG): container finished" podID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerID="2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28" exitCode=0
Jul 01 15:29:19.023591 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.023406    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"
Jul 01 15:29:19.023591 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.023431    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerDied","Data":"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"}
Jul 01 15:29:19.023591 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.023516    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb" event={"ID":"35e1c3d5-96d6-4d20-a44d-94df98d50628","Type":"ContainerDied","Data":"eab58021635bd4e207307c6543b0ce66e39dbba34e5099f19cdd282732dfaaec"}
Jul 01 15:29:19.023591 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.023558    2541 scope.go:117] "RemoveContainer" containerID="65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"
Jul 01 15:29:19.034312 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.034292    2541 scope.go:117] "RemoveContainer" containerID="2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"
Jul 01 15:29:19.044408 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.044380    2541 scope.go:117] "RemoveContainer" containerID="65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"
Jul 01 15:29:19.044871 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:19.044831    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168\": container with ID starting with 65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168 not found: ID does not exist" containerID="65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"
Jul 01 15:29:19.044980 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.044879    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168"} err="failed to get container status \"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168\": rpc error: code = NotFound desc = could not find container \"65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168\": container with ID starting with 65c9a51f9d33267b8fe1072ccc4345c52f0f058a8a63bb7e55e6258b542d4168 not found: ID does not exist"
Jul 01 15:29:19.044980 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.044905    2541 scope.go:117] "RemoveContainer" containerID="2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"
Jul 01 15:29:19.045237 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:19.045218    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28\": container with ID starting with 2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28 not found: ID does not exist" containerID="2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"
Jul 01 15:29:19.045280 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.045253    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28"} err="failed to get container status \"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28\": rpc error: code = NotFound desc = could not find container \"2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28\": container with ID starting with 2f43037cf6bdb5b13800fff6c123688396e41fc7ead6bd55e97bf154f9dcea28 not found: ID does not exist"
Jul 01 15:29:19.050332 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.050304    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:29:19.057526 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.057464    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5851-predictor-95c7775db-wcjpb"]
Jul 01 15:29:19.727481 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:19.727447    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" path="/var/lib/kubelet/pods/35e1c3d5-96d6-4d20-a44d-94df98d50628/volumes"
Jul 01 15:29:24.056413 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:24.056380    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" event={"ID":"4c00e6ce-bdaa-4938-a53a-081c7fd577c5","Type":"ContainerStarted","Data":"2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64"}
Jul 01 15:29:24.056863 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:24.056423    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" event={"ID":"4c00e6ce-bdaa-4938-a53a-081c7fd577c5","Type":"ContainerStarted","Data":"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"}
Jul 01 15:29:24.076387 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:24.076322    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" podStartSLOduration=1.360758345 podStartE2EDuration="8.076299273s" totalImagesPullingTime="6.715540928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:29:24.073215359 +0000 UTC m=+3012.789600118" watchObservedRunningTime="2026-07-01 15:29:24.076299273 +0000 UTC m=+3012.792684035"
Jul 01 15:29:45.172699 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:45.172665    2541 generic.go:356] "Generic (PLEG): container finished" podID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerID="5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155" exitCode=0
Jul 01 15:29:45.173179 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:45.172734    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" event={"ID":"4c00e6ce-bdaa-4938-a53a-081c7fd577c5","Type":"ContainerDied","Data":"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"}
Jul 01 15:29:45.173179 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:45.173109    2541 scope.go:117] "RemoveContainer" containerID="5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"
Jul 01 15:29:45.834746 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:45.834712    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4cr6f_must-gather-gsz6d_4c00e6ce-bdaa-4938-a53a-081c7fd577c5/gather/0.log"
Jul 01 15:29:46.328936 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:46.328899    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:46.473129 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:46.473102    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:46.588803 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:46.588721    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:49.407330 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:49.407302    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mwx8j_9143e78e-04cd-4a6b-8fd3-a25bf0aa0836/global-pull-secret-syncer/0.log"
Jul 01 15:29:49.467345 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:49.467317    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log"
Jul 01 15:29:49.568907 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:49.568879    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-214.ec2.internal_4b33c2a96ad53e3015792631c1ed9d0e/haproxy/0.log"
Jul 01 15:29:51.287910 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.287867    2541 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-4cr6f/must-gather-gsz6d"]
Jul 01 15:29:51.288402 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.288134    2541 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-4cr6f/must-gather-gsz6d" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="copy" containerID="cri-o://2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64" gracePeriod=2
Jul 01 15:29:51.290378 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:51.290321    2541 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gsz6d\" is forbidden: User \"system:node:ip-10-0-133-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4cr6f\": no relationship found between node 'ip-10-0-133-214.ec2.internal' and this object" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:51.291464 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.291441    2541 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-4cr6f/must-gather-gsz6d"]
Jul 01 15:29:51.525160 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.525138    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4cr6f_must-gather-gsz6d_4c00e6ce-bdaa-4938-a53a-081c7fd577c5/copy/0.log"
Jul 01 15:29:51.526731 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.526714    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:51.528701 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:51.528677    2541 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gsz6d\" is forbidden: User \"system:node:ip-10-0-133-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4cr6f\": no relationship found between node 'ip-10-0-133-214.ec2.internal' and this object" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:51.672090 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.672049    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output\") pod \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") "
Jul 01 15:29:51.672270 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.672121    2541 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm2k4\" (UniqueName: \"kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4\") pod \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\" (UID: \"4c00e6ce-bdaa-4938-a53a-081c7fd577c5\") "
Jul 01 15:29:51.673616 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.673583    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "4c00e6ce-bdaa-4938-a53a-081c7fd577c5" (UID: "4c00e6ce-bdaa-4938-a53a-081c7fd577c5"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 15:29:51.674482 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.674458    2541 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4" (OuterVolumeSpecName: "kube-api-access-mm2k4") pod "4c00e6ce-bdaa-4938-a53a-081c7fd577c5" (UID: "4c00e6ce-bdaa-4938-a53a-081c7fd577c5"). InnerVolumeSpecName "kube-api-access-mm2k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:29:51.723746 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:51.723709    2541 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gsz6d\" is forbidden: User \"system:node:ip-10-0-133-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4cr6f\": no relationship found between node 'ip-10-0-133-214.ec2.internal' and this object" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:51.725432 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.725410    2541 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" path="/var/lib/kubelet/pods/4c00e6ce-bdaa-4938-a53a-081c7fd577c5/volumes"
Jul 01 15:29:51.773741 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.773686    2541 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-must-gather-output\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:29:51.773741 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:51.773736    2541 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm2k4\" (UniqueName: \"kubernetes.io/projected/4c00e6ce-bdaa-4938-a53a-081c7fd577c5-kube-api-access-mm2k4\") on node \"ip-10-0-133-214.ec2.internal\" DevicePath \"\""
Jul 01 15:29:52.121218 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.121109    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:52.206690 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.206667    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4cr6f_must-gather-gsz6d_4c00e6ce-bdaa-4938-a53a-081c7fd577c5/copy/0.log"
Jul 01 15:29:52.207030 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.207012    2541 generic.go:356] "Generic (PLEG): container finished" podID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerID="2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64" exitCode=143
Jul 01 15:29:52.207083 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.207066    2541 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cr6f/must-gather-gsz6d"
Jul 01 15:29:52.207127 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.207115    2541 scope.go:117] "RemoveContainer" containerID="2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64"
Jul 01 15:29:52.215214 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.215182    2541 scope.go:117] "RemoveContainer" containerID="5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"
Jul 01 15:29:52.220270 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.220251    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:29:52.227934 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.227916    2541 scope.go:117] "RemoveContainer" containerID="2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64"
Jul 01 15:29:52.228174 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:52.228156    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64\": container with ID starting with 2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64 not found: ID does not exist" containerID="2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64"
Jul 01 15:29:52.228236 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.228182    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64"} err="failed to get container status \"2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64\": rpc error: code = NotFound desc = could not find container \"2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64\": container with ID starting with 2134043b4fd0b16f148a908ff76764899ddfa4051bc9c810a21a5d909377ad64 not found: ID does not exist"
Jul 01 15:29:52.228236 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.228200    2541 scope.go:117] "RemoveContainer" containerID="5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"
Jul 01 15:29:52.228419 ip-10-0-133-214 kubenswrapper[2541]: E0701 15:29:52.228402    2541 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155\": container with ID starting with 5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155 not found: ID does not exist" containerID="5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"
Jul 01 15:29:52.228463 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.228426    2541 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155"} err="failed to get container status \"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155\": rpc error: code = NotFound desc = could not find container \"5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155\": container with ID starting with 5a0ceb82c6bcc6d9f4a03acbf30da1abcec6dc8b5cf3cc30268a0d6385751155 not found: ID does not exist"
Jul 01 15:29:52.499195 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.499168    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/alertmanager/0.log"
Jul 01 15:29:52.547284 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.547257    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/config-reloader/0.log"
Jul 01 15:29:52.586545 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.586515    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/kube-rbac-proxy-web/0.log"
Jul 01 15:29:52.634761 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.634735    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/kube-rbac-proxy/0.log"
Jul 01 15:29:52.676859 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.676825    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/kube-rbac-proxy-metric/0.log"
Jul 01 15:29:52.721575 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.721544    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/prom-label-proxy/0.log"
Jul 01 15:29:52.782338 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.782218    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1157b362-a71a-4faa-9201-e0fa88fa5c05/init-config-reloader/0.log"
Jul 01 15:29:52.865124 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:52.865098    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-klcwt_0858db31-5603-4785-b3ba-6dc0a6d3e7d7/cluster-monitoring-operator/0.log"
Jul 01 15:29:53.103986 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.103907    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86f44597f8-lw42k_2a63bef6-9b82-4748-8bcc-c91a8fa0ec2e/metrics-server/0.log"
Jul 01 15:29:53.148246 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.148214    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-fbjnc_fc507241-b77e-4c75-9a91-af447ff5f2c0/monitoring-plugin/0.log"
Jul 01 15:29:53.398319 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.398237    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqm7n_ee6ca283-3584-4392-af96-b83cf6a4b97a/node-exporter/0.log"
Jul 01 15:29:53.453855 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.453811    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqm7n_ee6ca283-3584-4392-af96-b83cf6a4b97a/kube-rbac-proxy/0.log"
Jul 01 15:29:53.498715 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.498688    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqm7n_ee6ca283-3584-4392-af96-b83cf6a4b97a/init-textfile/0.log"
Jul 01 15:29:53.661041 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.661021    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-d5g9f_383a8cfa-3e3e-4a5a-a5c8-91cca1468071/kube-rbac-proxy-main/0.log"
Jul 01 15:29:53.702863 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.702817    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-d5g9f_383a8cfa-3e3e-4a5a-a5c8-91cca1468071/kube-rbac-proxy-self/0.log"
Jul 01 15:29:53.737733 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:53.737696    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-d5g9f_383a8cfa-3e3e-4a5a-a5c8-91cca1468071/openshift-state-metrics/0.log"
Jul 01 15:29:54.128306 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.128209    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb57c58c-5twzt_d913a3b9-c2eb-4e2c-b690-a202daef584f/telemeter-client/0.log"
Jul 01 15:29:54.156123 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.156092    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb57c58c-5twzt_d913a3b9-c2eb-4e2c-b690-a202daef584f/reload/0.log"
Jul 01 15:29:54.191823 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.191797    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb57c58c-5twzt_d913a3b9-c2eb-4e2c-b690-a202daef584f/kube-rbac-proxy/0.log"
Jul 01 15:29:54.236391 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.236361    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/thanos-query/0.log"
Jul 01 15:29:54.260643 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.260615    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/kube-rbac-proxy-web/0.log"
Jul 01 15:29:54.288125 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.288107    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/kube-rbac-proxy/0.log"
Jul 01 15:29:54.314896 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.314869    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/prom-label-proxy/0.log"
Jul 01 15:29:54.352287 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.352265    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/kube-rbac-proxy-rules/0.log"
Jul 01 15:29:54.401029 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:54.400958    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9b98ff569-q7qhl_a5c010ab-6ede-47ce-8e75-e2c1f2cc9aec/kube-rbac-proxy-metrics/0.log"
Jul 01 15:29:55.813411 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:55.813320    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-kwj9r_0cf4325c-d583-4906-98e8-ac0332a96536/networking-console-plugin/0.log"
Jul 01 15:29:56.415390 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:56.415354    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/1.log"
Jul 01 15:29:56.419351 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:56.419329    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8bwx_ef110078-ba7d-4469-9fc8-3ae18fa17f51/console-operator/2.log"
Jul 01 15:29:57.402659 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.402628    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-f4vss_d309b969-9fb4-412d-b9c9-b793a68700c0/volume-data-source-validator/0.log"
Jul 01 15:29:57.727189 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727161    2541 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"]
Jul 01 15:29:57.727502 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727490    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="copy"
Jul 01 15:29:57.727550 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727504    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="copy"
Jul 01 15:29:57.727550 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727519    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="gather"
Jul 01 15:29:57.727550 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727525    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="gather"
Jul 01 15:29:57.727550 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727541    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container"
Jul 01 15:29:57.727550 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727547    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727558    2541 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kube-rbac-proxy"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727564    2541 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kube-rbac-proxy"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727622    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kube-rbac-proxy"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727634    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="gather"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727641    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c00e6ce-bdaa-4938-a53a-081c7fd577c5" containerName="copy"
Jul 01 15:29:57.727759 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.727648    2541 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e1c3d5-96d6-4d20-a44d-94df98d50628" containerName="kserve-container"
Jul 01 15:29:57.731330 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.731307    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.734424 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.734399    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wp8cx\"/\"openshift-service-ca.crt\""
Jul 01 15:29:57.734546 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.734437    2541 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wp8cx\"/\"kube-root-ca.crt\""
Jul 01 15:29:57.735545 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.735525    2541 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wp8cx\"/\"default-dockercfg-ddj5c\""
Jul 01 15:29:57.742306 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.742282    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"]
Jul 01 15:29:57.827752 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.827715    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-proc\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.828260 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.828224    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngff7\" (UniqueName: \"kubernetes.io/projected/6b0ec958-128a-4284-ab5c-30d20da02f74-kube-api-access-ngff7\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.828424 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.828293    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-sys\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.828501 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.828430    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-lib-modules\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.828501 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.828457    2541 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-podres\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.929716 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929674    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngff7\" (UniqueName: \"kubernetes.io/projected/6b0ec958-128a-4284-ab5c-30d20da02f74-kube-api-access-ngff7\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.929716 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929723    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-sys\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929758    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-lib-modules\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929773    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-podres\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929809    2541 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-proc\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929920    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-sys\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929919    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-proc\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929959    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-podres\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.930037 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.929969    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b0ec958-128a-4284-ab5c-30d20da02f74-lib-modules\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:57.941365 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:57.941327    2541 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngff7\" (UniqueName: \"kubernetes.io/projected/6b0ec958-128a-4284-ab5c-30d20da02f74-kube-api-access-ngff7\") pod \"perf-node-gather-daemonset-j978f\" (UID: \"6b0ec958-128a-4284-ab5c-30d20da02f74\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:58.041618 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.041516    2541 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:58.177628 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.177600    2541 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"]
Jul 01 15:29:58.179915 ip-10-0-133-214 kubenswrapper[2541]: W0701 15:29:58.179883    2541 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6b0ec958_128a_4284_ab5c_30d20da02f74.slice/crio-7959310506c0a58a61dd7705936d852ea19403ea6df996ab5962ef5e3a02cfea WatchSource:0}: Error finding container 7959310506c0a58a61dd7705936d852ea19403ea6df996ab5962ef5e3a02cfea: Status 404 returned error can't find the container with id 7959310506c0a58a61dd7705936d852ea19403ea6df996ab5962ef5e3a02cfea
Jul 01 15:29:58.237773 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.237741    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f" event={"ID":"6b0ec958-128a-4284-ab5c-30d20da02f74","Type":"ContainerStarted","Data":"7959310506c0a58a61dd7705936d852ea19403ea6df996ab5962ef5e3a02cfea"}
Jul 01 15:29:58.450602 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.450577    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlnd6_2ca78877-45e9-427b-9ca5-ead578a8d572/dns/0.log"
Jul 01 15:29:58.474024 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.473988    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlnd6_2ca78877-45e9-427b-9ca5-ead578a8d572/kube-rbac-proxy/0.log"
Jul 01 15:29:58.497576 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:58.497542    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rb9zw_1cb87f50-27ec-4332-bb8d-5f33ffa8fa45/dns-node-resolver/0.log"
Jul 01 15:29:59.065762 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:59.065730    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jjvg_31c63ed0-9e3f-4e22-8844-f28c0d5b9425/node-ca/0.log"
Jul 01 15:29:59.245318 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:59.245286    2541 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f" event={"ID":"6b0ec958-128a-4284-ab5c-30d20da02f74","Type":"ContainerStarted","Data":"9f764e2c3e9dc1917bcff31034cbb60a1c22e11a146dbc24fe0c7eb58b93c992"}
Jul 01 15:29:59.245505 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:59.245388    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:29:59.262932 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:59.262882    2541 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f" podStartSLOduration=2.2628636970000002 podStartE2EDuration="2.262863697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 15:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:29:59.261485808 +0000 UTC m=+3047.977870566" watchObservedRunningTime="2026-07-01 15:29:59.262863697 +0000 UTC m=+3047.979248446"
Jul 01 15:29:59.925697 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:29:59.925650    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b66db9b69-6qgkk_70317fe5-e0e4-43f4-a19a-483c4b33fd40/router/0.log"
Jul 01 15:30:00.315562 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.315476    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qhzkw_357a4182-6b1f-4064-9a16-d280230a9061/serve-healthcheck-canary/0.log"
Jul 01 15:30:00.697303 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.697266    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-7qj2r_b2b0c7e8-e90e-4093-ae0f-82e75413c8cf/insights-operator/0.log"
Jul 01 15:30:00.697861 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.697827    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-7qj2r_b2b0c7e8-e90e-4093-ae0f-82e75413c8cf/insights-operator/1.log"
Jul 01 15:30:00.720077 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.720051    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnf6_77a11177-c341-4312-be30-67977ad182ea/kube-rbac-proxy/0.log"
Jul 01 15:30:00.746220 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.746192    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnf6_77a11177-c341-4312-be30-67977ad182ea/exporter/0.log"
Jul 01 15:30:00.770651 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:00.770623    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6gnf6_77a11177-c341-4312-be30-67977ad182ea/extractor/0.log"
Jul 01 15:30:03.085459 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:03.085362    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-zcgz5_a1110e7d-eef9-4ff4-90bf-d8578c0d4597/manager/0.log"
Jul 01 15:30:03.250087 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:03.250055    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-lkjcs_46f9807a-a638-4af3-bc62-b1c0d4129bff/s3-init/0.log"
Jul 01 15:30:03.265625 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:03.265596    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-lkjcs_46f9807a-a638-4af3-bc62-b1c0d4129bff/download-hf-model/0.log"
Jul 01 15:30:05.260501 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:05.260474    2541 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-j978f"
Jul 01 15:30:07.750817 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:07.750781    2541 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5t2l_1de49db7-4331-4304-b09e-300573ca8724/konnectivity-agent/0.log" limit=1024
Jul 01 15:30:08.001698 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:08.001615    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-r8q4x_61c6cdfc-f872-43a1-ace1-ee78a7c8b5af/kube-storage-version-migrator-operator/1.log"
Jul 01 15:30:08.002464 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:08.002444    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-r8q4x_61c6cdfc-f872-43a1-ace1-ee78a7c8b5af/kube-storage-version-migrator-operator/0.log"
Jul 01 15:30:09.038191 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.038159    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/kube-multus-additional-cni-plugins/0.log"
Jul 01 15:30:09.059427 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.059400    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/egress-router-binary-copy/0.log"
Jul 01 15:30:09.082920 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.082895    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/cni-plugins/0.log"
Jul 01 15:30:09.102902 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.102875    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/bond-cni-plugin/0.log"
Jul 01 15:30:09.125712 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.125685    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/routeoverride-cni/0.log"
Jul 01 15:30:09.146091 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.146030    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/whereabouts-cni-bincopy/0.log"
Jul 01 15:30:09.167514 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.167492    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fqlzc_bb43336f-c51e-4d6f-8708-89f7e9713860/whereabouts-cni/0.log"
Jul 01 15:30:09.410094 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.410068    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k998v_5e28982b-cd07-4630-92f5-0f4ce06b6ce1/kube-multus/0.log"
Jul 01 15:30:09.506405 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.506374    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p5s2b_3500fb9f-71cb-4c7d-a1f7-9d58da2185eb/network-metrics-daemon/0.log"
Jul 01 15:30:09.527145 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:09.527112    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p5s2b_3500fb9f-71cb-4c7d-a1f7-9d58da2185eb/kube-rbac-proxy/0.log"
Jul 01 15:30:10.864621 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.864584    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-controller/0.log"
Jul 01 15:30:10.884735 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.884707    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/1.log"
Jul 01 15:30:10.884926 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.884738    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovn-acl-logging/0.log"
Jul 01 15:30:10.908074 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.908050    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/kube-rbac-proxy-node/0.log"
Jul 01 15:30:10.930691 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.930666    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 15:30:10.951585 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.951565    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/northd/0.log"
Jul 01 15:30:10.972957 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.972928    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/nbdb/0.log"
Jul 01 15:30:10.996435 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:10.996407    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/sbdb/0.log"
Jul 01 15:30:11.043811 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:11.043776    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zh22p_8b2ff6ec-f4d3-4760-8695-385d1cbfab3d/ovnkube-controller/0.log"
Jul 01 15:30:12.045755 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:12.045722    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-zgshx_b9de1d7e-87fd-4857-997c-51493ced5359/check-endpoints/0.log"
Jul 01 15:30:12.090059 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:12.090024    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-dz29r_0c67366f-5b37-478a-911f-0d0fe521267b/network-check-target-container/0.log"
Jul 01 15:30:13.052009 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:13.051981    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-64w57_d841bd2a-2624-4f19-9421-b2e0eed2a1d2/iptables-alerter/0.log"
Jul 01 15:30:13.769025 ip-10-0-133-214 kubenswrapper[2541]: I0701 15:30:13.768998    2541 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-bfdpj_e9079c38-8718-4862-a6b6-c266032405c0/tuned/0.log"