Jul 07 15:43:46.616689 ip-10-0-141-42 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.093190 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097267 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097280 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097284 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097287 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097290 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097292 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097295 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.224106 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097298 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097301 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097303 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097306 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097309 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097311 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097314 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097316 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097319 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097322 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097325 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097328 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097330 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097333 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097339 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097341 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097344 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097347 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097349 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097352 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.228261 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097355 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097357 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097360 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097362 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097365 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097367 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097370 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097373 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097393 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097396 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097399 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097401 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097404 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097407 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097410 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097413 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097416 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097419 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097421 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097424 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.229059 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097427 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097432 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097435 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097438 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097441 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097444 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097447 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097450 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097452 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097455 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097458 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097461 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097464 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097467 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097469 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097472 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097474 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097478 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097482 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.230918 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097485 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097489 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097493 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097496 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097499 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097502 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097504 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097507 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097510 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097513 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097515 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097518 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097521 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097523 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097526 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097529 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097531 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097534 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097537 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097539 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.231764 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097542 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097545 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097548 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097550 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097554 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097556 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097559 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097561 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097564 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097567 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097570 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097572 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.097575 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098276 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098282 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098285 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098288 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098290 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098293 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098297 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.234887 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098299 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098302 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098305 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098307 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098310 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098312 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098315 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098317 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098320 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098322 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098325 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098327 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098330 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098334 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098339 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098343 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098346 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098348 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098351 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098354 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.235746 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098356 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098359 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098361 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098364 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098366 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098369 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098372 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098393 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098396 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098399 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098401 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098404 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098407 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098409 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098412 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098414 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098417 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098420 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098422 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098425 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.237940 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098427 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098431 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098433 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098436 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098439 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098441 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098444 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098448 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098451 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098454 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098457 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098459 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098462 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098465 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098467 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098470 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098472 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098475 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098477 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098480 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.238827 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098482 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098485 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098488 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098490 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098493 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098496 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098498 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098501 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098503 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098505 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098508 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098510 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098515 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098518 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098520 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098523 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098525 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098528 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.239710 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098531 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098535 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098542 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098545 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098547 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098551 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098554 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098557 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098560 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098562 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098565 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098567 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098570 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098572 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098951 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098956 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098958 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098961 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098964 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.240650 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098966 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098969 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098972 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098976 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098979 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098981 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098984 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098987 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098989 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098992 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098995 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.098998 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099000 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099005 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099008 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099013 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099016 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099019 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099022 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.244121 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099025 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099028 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099030 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099033 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099036 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099038 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099041 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099044 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099047 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099049 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099052 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099054 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099057 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099059 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099062 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099065 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099067 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099070 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099072 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099075 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.245431 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099077 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099080 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099083 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099085 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099088 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099090 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099093 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099096 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099100 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099107 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099111 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099114 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099116 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099119 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099122 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099124 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099128 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099131 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099134 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099137 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.247636 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099139 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099142 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099145 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099147 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099151 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099154 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099157 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099161 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099163 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099166 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099168 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099171 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099173 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099176 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099178 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099181 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099184 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099187 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099189 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.248526 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099192 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099196 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099200 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099203 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099206 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099208 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099211 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099213 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099216 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099218 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099221 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099223 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099226 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099228 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099231 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099234 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099306 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099317 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099327 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099333 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099339 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:43:47.261043 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099343 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099348 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099352 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099355 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099358 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099361 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099365 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099368 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099371 2562 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099391 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099395 2562 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099398 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099403 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099410 2562 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099412 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099417 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099420 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099423 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099427 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099430 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099433 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099437 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099440 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099443 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:43:47.261906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099446 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099449 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099452 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099456 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099459 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099462 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099464 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099467 2562 flags.go:64] FLAG: --enable-server="true" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099470 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099476 2562 flags.go:64] FLAG: --event-burst="100" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099479 2562 flags.go:64] FLAG: --event-qps="50" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099482 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099485 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099488 2562 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099493 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099495 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099499 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099502 2562 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099504 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099508 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099512 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099516 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099519 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099522 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099527 2562 flags.go:64] FLAG: --feature-gates="" Jul 07 15:43:47.264151 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099531 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099534 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099537 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099540 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099543 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099547 2562 flags.go:64] FLAG: --help="false" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099549 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-141-42.ec2.internal" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099553 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099556 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099559 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099562 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099566 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099569 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099572 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099574 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099577 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099580 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099583 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099586 2562 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099589 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099592 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099595 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099598 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099601 2562 flags.go:64] FLAG: --lock-file="" Jul 07 15:43:47.265031 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099603 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099606 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099609 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099616 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099620 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099623 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099626 2562 flags.go:64] FLAG: --logging-format="text" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099629 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099633 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099636 2562 flags.go:64] FLAG: --manifest-url="" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099638 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099646 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099649 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099653 2562 flags.go:64] FLAG: --max-pods="110" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099656 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099659 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099662 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099665 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099668 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099671 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099674 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099681 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099684 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099687 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:43:47.372281 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099690 2562 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:43:47.280301 ip-10-0-141-42 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099693 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099699 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099702 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099705 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099708 2562 flags.go:64] FLAG: --port="10250" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099711 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099714 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb6ee6b190f0b285" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099718 2562 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099721 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099723 2562 flags.go:64] FLAG: --register-node="true" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099726 2562 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099732 2562 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099735 2562 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099738 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099741 2562 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099745 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099749 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099752 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099755 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099758 2562 flags.go:64] FLAG: --runonce="false" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099761 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099764 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099767 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099769 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:43:47.373647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099772 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099775 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099778 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099781 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099784 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099787 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099790 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099793 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099796 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099799 2562 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099801 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099807 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099810 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099812 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099816 2562 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099819 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099822 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099825 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099828 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099834 2562 flags.go:64] FLAG: --v="2" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099838 2562 flags.go:64] FLAG: --version="false" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099842 2562 flags.go:64] FLAG: --vmodule="" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099846 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.099850 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099992 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.376721 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.099997 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100000 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100003 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100006 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100009 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100011 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100014 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100016 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100019 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100021 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100024 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100026 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100028 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100031 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100033 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100036 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100038 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100041 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100043 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100046 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.377567 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100048 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100051 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100053 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100056 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100058 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100061 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100065 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100068 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100071 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100073 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100076 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100078 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100083 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100087 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100090 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100093 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100095 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100098 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100101 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.380993 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100105 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100108 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100111 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100113 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100116 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100118 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100121 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100123 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100126 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100128 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100131 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100134 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100136 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100139 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100142 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100154 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100156 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100159 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100161 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100166 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.381838 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100169 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100171 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100174 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100177 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100179 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100182 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100185 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100189 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100191 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100194 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100197 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100199 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100202 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100205 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100207 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100210 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100212 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100215 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100217 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100220 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.384913 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100223 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100226 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100230 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100233 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100236 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100239 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100242 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100244 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100247 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100250 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100252 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100259 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100262 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100265 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100267 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100271 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100273 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100276 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.385794 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.100279 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.101053 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.110475 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.110497 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.112288 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.112303 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112413 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112420 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112426 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112430 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112435 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112440 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112443 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112449 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112453 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.391959 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112457 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112462 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112466 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112470 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112474 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112477 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112482 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112486 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112490 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112494 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112498 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112502 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112506 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112509 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112513 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112518 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112525 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112529 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112534 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.392791 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112538 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112541 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112545 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112551 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112556 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112560 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112564 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112568 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112572 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112576 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112579 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112583 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112587 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112591 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112594 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112598 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112602 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112606 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112611 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112615 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.398003 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112622 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112627 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112631 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112635 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112639 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112643 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112647 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112650 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112654 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112658 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112662 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112666 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112670 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112674 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112678 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112683 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112686 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112691 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112694 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112698 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.398855 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112704 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112709 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112713 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112717 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112721 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112726 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112730 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112735 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112739 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112743 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112747 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112752 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112757 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112761 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112765 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112769 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112772 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112776 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112780 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112784 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.402301 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112787 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112791 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112816 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112823 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112828 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112833 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112837 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112840 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112844 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112849 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.112853 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.112859 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113026 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113033 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113037 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:43:47.403162 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113042 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113047 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113050 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113055 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113059 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113064 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113067 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113071 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113075 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113082 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113085 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113090 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113096 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113100 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113103 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113107 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113111 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113115 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113118 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113122 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:43:47.406225 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113125 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113129 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113133 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113137 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113140 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113145 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113148 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113152 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113156 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113160 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113164 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113167 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113171 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113174 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113178 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113182 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113186 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113189 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113193 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113197 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:43:47.407119 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113201 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113206 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113211 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113216 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113221 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113225 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113229 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113233 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113237 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113241 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113245 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113251 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113255 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113260 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113264 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113268 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113272 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113275 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113279 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:43:47.411463 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113283 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113287 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113291 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113294 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113298 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113302 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113306 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113310 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113314 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113318 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113322 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113325 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113330 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113333 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113338 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113342 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113347 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113354 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113357 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113361 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:43:47.412341 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113365 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113369 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113372 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113391 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113395 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113398 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113401 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113405 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113409 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113413 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113417 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113422 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113427 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113433 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113437 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113442 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:43:47.518515 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:47.113447 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.113453 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.113601 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.217919 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.219895 2562 server.go:1019] "Starting client certificate rotation" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.220002 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.220048 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.222946 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.224476 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.237582 2562 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.242358 2562 log.go:25] "Validated CRI v1 image API" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.244350 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:43:47.519289 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.248738 2562 fs.go:135] Filesystem UUIDs: map[58c5fbb7-16dc-4bfd-995c-db5229ac3ecb:/dev/nvme0n1p3 7425f51d-ca99-4438-a953-c4198afc6a6a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 15:43:47.522125 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.248756 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.254455 2562 manager.go:216] Machine: {Timestamp:2026-07-07 15:43:47.253889016 +0000 UTC m=+0.484955281 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100367 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23a9cee20ed2a57009f08734577da7 SystemUUID:ec23a9ce-e20e-d2a5-7009-f08734577da7 BootID:435a7897-92a2-42be-80d3-6f58d485fb82 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:f8:9a:a2:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:f8:9a:a2:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:ac:92:b2:50:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.254569 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.254680 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258012 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258043 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258547 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:43:47.522535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258835 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258847 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.258855 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.260512 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.262422 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.266873 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.266988 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267019 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267031 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267045 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267054 2562 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267062 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267477 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.267496 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.271501 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.274079 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.274106 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275622 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275670 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275677 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275682 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275691 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275696 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275704 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275709 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275715 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275721 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275736 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.275742 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.276649 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.276657 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.279650 2562 server.go:1268] "Started kubelet" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.281002 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.283134 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.283186 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.284089 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.284719 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.285360 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.285501 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:43:47.524981 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.285502 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.291857 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-42.ec2.internal.18c00c58a392005c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-42.ec2.internal,UID:ip-10-0-141-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-42.ec2.internal,},FirstTimestamp:2026-07-07 15:43:47.279626332 +0000 UTC m=+0.510692598,LastTimestamp:2026-07-07 15:43:47.279626332 +0000 UTC m=+0.510692598,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-42.ec2.internal,}" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.293667 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.297524 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.297902 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.300347 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.301146 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.301153 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.301304 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.301346 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.301350 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302158 2562 factory.go:55] Registering systemd factory Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302170 2562 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302338 2562 factory.go:153] Registering CRI-O factory Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302344 2562 factory.go:223] Registration of the crio container factory successfully Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302403 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302425 2562 factory.go:103] Registering Raw factory Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302447 2562 manager.go:1196] Started watching for new ooms in manager Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.302764 2562 manager.go:349] Starting recovery of all containers Jul 07 15:43:47.526225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.304187 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:43:47.795331 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.793471 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:47.795331 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.795011 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:43:47.795331 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.795087 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:43:47.795331 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.795155 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b2gf7" Jul 07 15:43:47.795331 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.795191 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b2gf7" Jul 07 15:43:47.798228 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.798205 2562 manager.go:354] Recovery completed Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.804192 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.806485 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.806514 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.806527 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.807225 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.807225 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.807281 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.807300 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.808013 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.808045 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.808059 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.809647 2562 policy_none.go:49] "None policy: Start" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.809662 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.809673 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:43:47.815129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.811881 2562 policy_none.go:47] "Start" Jul 07 15:43:47.867732 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.867543 2562 manager.go:337] "Starting Device Plugin manager" Jul 07 15:43:47.867853 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.867778 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:43:47.867853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.867794 2562 server.go:87] "Starting device plugin registration server" Jul 07 15:43:47.868088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.868073 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:43:47.868229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.868090 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:43:47.868229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.868189 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:43:47.868324 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.868283 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:43:47.868324 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.868295 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:43:47.869970 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.869949 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:43:47.870063 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.870000 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:47.919717 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.919683 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:43:47.922050 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.919734 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:43:47.922050 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.919759 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:43:47.922050 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.919799 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:43:47.922941 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.922924 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:43:47.970604 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.970575 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:47.971639 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.971620 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:47.971734 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.971653 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:47.971734 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.971663 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:47.971734 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.971690 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-42.ec2.internal" Jul 07 15:43:47.991545 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:47.991512 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-42.ec2.internal" Jul 07 15:43:47.991545 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:47.991544 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-42.ec2.internal\": node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:48.020360 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.020318 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal"] Jul 07 15:43:48.020549 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.020423 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:48.022539 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.022519 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:48.022653 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.022560 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:48.022653 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.022575 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:48.025074 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.025058 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:48.025199 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.025181 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.025257 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.025217 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:48.026012 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.025997 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:48.026095 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.026029 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:48.026095 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.026044 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:48.026095 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.026000 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:48.026241 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.026109 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:48.026241 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.026119 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:48.027394 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:48.027369 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:48.028556 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.028540 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.028649 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.028569 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:43:48.029308 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.029289 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:43:48.029373 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.029328 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:43:48.029373 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.029343 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:43:48.052342 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:48.052268 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-42.ec2.internal\" not found" node="ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.057140 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:48.057116 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-42.ec2.internal\" not found" node="ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.128131 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:48.128097 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:48.194882 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.194842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.194962 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.194891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.194962 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.194919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db9c52c440a1356e3cef0bcdda630a66-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.ec2.internal\" (UID: \"db9c52c440a1356e3cef0bcdda630a66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.218904 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.218854 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:43:48.219083 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.219060 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:43:48.228722 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:48.228696 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-42.ec2.internal\" not found" Jul 07 15:43:48.294566 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.294538 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:43:48.295650 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.295702 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.295702 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db9c52c440a1356e3cef0bcdda630a66-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.ec2.internal\" (UID: \"db9c52c440a1356e3cef0bcdda630a66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.295772 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295742 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db9c52c440a1356e3cef0bcdda630a66-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.ec2.internal\" (UID: \"db9c52c440a1356e3cef0bcdda630a66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.295814 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295802 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.295853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.295832 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff025dfd860c0fee39bf6f7d27498ac0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal\" (UID: \"ff025dfd860c0fee39bf6f7d27498ac0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.300475 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.300460 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:43:48.302113 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.302100 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.324512 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.324452 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:43:48.346715 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.346684 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:43:48.348322 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.348306 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.355135 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.355118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.360339 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.360139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" Jul 07 15:43:48.367526 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.367498 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:43:48.458636 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.458603 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xwwdd" Jul 07 15:43:48.471979 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.471953 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xwwdd" Jul 07 15:43:48.720408 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.720300 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:43:48.800736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.800670 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 15:38:47 +0000 UTC" deadline="2027-12-22 17:29:09.526211047 +0000 UTC" Jul 07 15:43:48.800736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.800724 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12793h45m20.725490549s" Jul 07 15:43:48.970162 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:48.970132 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:43:49.037488 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.037452 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff025dfd860c0fee39bf6f7d27498ac0.slice/crio-a08ba2c5c0be3b78f38dc141009cecfc7a08edc5bc5f9d38edb6bfdc2750df42 WatchSource:0}: Error finding container a08ba2c5c0be3b78f38dc141009cecfc7a08edc5bc5f9d38edb6bfdc2750df42: Status 404 returned error can't find the container with id a08ba2c5c0be3b78f38dc141009cecfc7a08edc5bc5f9d38edb6bfdc2750df42 Jul 07 15:43:49.045072 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.045045 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:43:49.050316 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.050292 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9c52c440a1356e3cef0bcdda630a66.slice/crio-c826bb5d39829e2f5bb3610dd112811befb22fbfe64bc5f248fb5924c3cbe98c WatchSource:0}: Error finding container c826bb5d39829e2f5bb3610dd112811befb22fbfe64bc5f248fb5924c3cbe98c: Status 404 returned error can't find the container with id c826bb5d39829e2f5bb3610dd112811befb22fbfe64bc5f248fb5924c3cbe98c Jul 07 15:43:49.271000 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.270966 2562 apiserver.go:52] "Watching apiserver" Jul 07 15:43:49.280001 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.279921 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:43:49.282417 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.282375 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nxs4p","openshift-multus/multus-hbpq8","openshift-network-operator/iptables-alerter-rwn98","kube-system/konnectivity-agent-mmwfv","kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal","openshift-image-registry/node-ca-t8q4v","openshift-multus/network-metrics-daemon-42jm9","openshift-network-diagnostics/network-check-target-qbdb7","openshift-ovn-kubernetes/ovnkube-node-w4qpz","kube-system/global-pull-secret-syncer-9mlh8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9","openshift-cluster-node-tuning-operator/tuned-5nk8h","openshift-dns/node-resolver-sphrk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal"] Jul 07 15:43:49.285250 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.285224 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.285371 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.285298 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:49.287494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.287476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.289785 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.289762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.291425 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.291405 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.291880 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.291866 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.292314 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.292293 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.292434 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.292297 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v7mck\"" Jul 07 15:43:49.292812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.292786 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.292926 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.292842 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:43:49.293065 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.293051 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.293615 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.293590 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f2qrz\"" Jul 07 15:43:49.294243 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.294173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.294718 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.294697 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:43:49.294816 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.294742 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:43:49.294925 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.294911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vptng\"" Jul 07 15:43:49.296618 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.296597 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.296754 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.296737 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.297367 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.297350 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:43:49.298489 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.298202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nb2d2\"" Jul 07 15:43:49.298489 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.298284 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300565 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300615 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300669 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300692 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:43:49.300807 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.300618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hswqs\"" Jul 07 15:43:49.301905 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.301884 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:43:49.301990 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.301884 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.303089 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.303073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.305616 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.305598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.306229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.306210 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:43:49.306350 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.306334 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.306653 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.306638 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.306742 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.306659 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggpxr\"" Jul 07 15:43:49.306803 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.306753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:43:49.307653 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.307632 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:43:49.307714 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.307666 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:43:49.307714 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.307692 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g6dfq\"" Jul 07 15:43:49.308171 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.308155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.309162 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crs92\" (UniqueName: \"kubernetes.io/projected/9b02124c-6de0-4c0e-83ed-914d2142429c-kube-api-access-crs92\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.309202 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-os-release\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309202 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-os-release\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.309299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b7b4e3-bc45-466d-905f-6e694553f73c-host-slash\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.309299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-agent-certs\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.309299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-cni-binary-copy\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309467 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b02124c-6de0-4c0e-83ed-914d2142429c-serviceca\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.309467 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-netd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309467 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02816b2d-54ac-4243-b607-535be7d41098-ovn-node-metrics-cert\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309467 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-konnectivity-ca\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.309467 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-k8s-cni-cncf-io\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-netns\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2crrf\" (UniqueName: \"kubernetes.io/projected/02816b2d-54ac-4243-b607-535be7d41098-kube-api-access-2crrf\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-hostroot\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-ovn\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-daemon-config\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.309659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-cnibin\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-var-lib-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-node-log\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6be354b0-ab22-49c7-826b-60a7886c2f21-hosts-file\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-kubelet\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-netns\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-env-overrides\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-socket-dir-parent\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-bin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-slash\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309939 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-bin\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.309987 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310099 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-script-lib\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nn8m\" (UniqueName: \"kubernetes.io/projected/639ebdf2-6bbe-42a5-8ade-322a278cad2d-kube-api-access-7nn8m\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b7b4e3-bc45-466d-905f-6e694553f73c-iptables-alerter-script\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310159 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-system-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-kubelet\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310258 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vf65g\"" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d7gf\" (UniqueName: \"kubernetes.io/projected/24b7b4e3-bc45-466d-905f-6e694553f73c-kube-api-access-9d7gf\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxhzf\" (UniqueName: \"kubernetes.io/projected/ab101994-6344-4297-ad93-deb48776b8e8-kube-api-access-dxhzf\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-cnibin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-conf-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-systemd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-config\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.310768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbcd6\" (UniqueName: \"kubernetes.io/projected/6be354b0-ab22-49c7-826b-60a7886c2f21-kube-api-access-cbcd6\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310533 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9hc66\"" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b02124c-6de0-4c0e-83ed-914d2142429c-host\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-etc-kubernetes\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be354b0-ab22-49c7-826b-60a7886c2f21-tmp-dir\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-multus-certs\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.310656 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkb6j\" (UniqueName: \"kubernetes.io/projected/be95066e-d25d-4913-bdfe-789f97dd3b31-kube-api-access-lkb6j\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-systemd-units\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-etc-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-log-socket\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-multus\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.310983 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.311449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.311084 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:43:49.312089 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.311317 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:43:49.313191 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.313168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.313264 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.313219 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:49.402922 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.402894 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:43:49.411199 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b02124c-6de0-4c0e-83ed-914d2142429c-host\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-etc-kubernetes\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be354b0-ab22-49c7-826b-60a7886c2f21-tmp-dir\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-kubernetes\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-etc-kubernetes\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411309 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b02124c-6de0-4c0e-83ed-914d2142429c-host\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.411371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-run\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-lib-modules\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411461 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-multus-certs\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkb6j\" (UniqueName: \"kubernetes.io/projected/be95066e-d25d-4913-bdfe-789f97dd3b31-kube-api-access-lkb6j\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411495 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-multus-certs\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-systemd-units\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-etc-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-log-socket\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-dbus\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411613 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-systemd-units\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-var-lib-kubelet\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-etc-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411656 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be354b0-ab22-49c7-826b-60a7886c2f21-tmp-dir\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-multus\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.411755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411695 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-log-socket\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-socket-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-multus\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crs92\" (UniqueName: \"kubernetes.io/projected/9b02124c-6de0-4c0e-83ed-914d2142429c-kube-api-access-crs92\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-os-release\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411840 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411854 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-os-release\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-os-release\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b7b4e3-bc45-466d-905f-6e694553f73c-host-slash\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-agent-certs\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411960 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b7b4e3-bc45-466d-905f-6e694553f73c-host-slash\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.411968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-cni-binary-copy\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-sys-fs\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdk6t\" (UniqueName: \"kubernetes.io/projected/2bafa631-7145-4626-92c2-e83051b58768-kube-api-access-qdk6t\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.412403 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b02124c-6de0-4c0e-83ed-914d2142429c-serviceca\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-netd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02816b2d-54ac-4243-b607-535be7d41098-ovn-node-metrics-cert\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-konnectivity-ca\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-netd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-k8s-cni-cncf-io\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-netns\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crrf\" (UniqueName: \"kubernetes.io/projected/02816b2d-54ac-4243-b607-535be7d41098-kube-api-access-2crrf\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412286 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412332 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-sys\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-hostroot\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-ovn\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-daemon-config\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412490 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-os-release\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-cnibin\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9b02124c-6de0-4c0e-83ed-914d2142429c-serviceca\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412573 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-cnibin\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412573 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-k8s-cni-cncf-io\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412635 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-hostroot\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412648 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-ovn\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412657 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-cni-binary-copy\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-run-netns\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412699 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-konnectivity-ca\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.412723 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-host\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.413948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412741 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-var-lib-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.412797 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:43:49.91277793 +0000 UTC m=+3.143844201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412726 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412812 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-var-lib-openvswitch\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-node-log\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-node-log\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-device-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412907 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-modprobe-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-conf\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-tmp\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.412989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6be354b0-ab22-49c7-826b-60a7886c2f21-hosts-file\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.414760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-kubelet\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-netns\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413046 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6be354b0-ab22-49c7-826b-60a7886c2f21-hosts-file\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-env-overrides\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413069 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-kubelet\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413073 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-run-netns\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-daemon-config\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-etc-tuned\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-socket-dir-parent\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-bin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-slash\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-bin\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.415552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-socket-dir-parent\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413330 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-script-lib\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nn8m\" (UniqueName: \"kubernetes.io/projected/639ebdf2-6bbe-42a5-8ade-322a278cad2d-kube-api-access-7nn8m\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-slash\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b7b4e3-bc45-466d-905f-6e694553f73c-iptables-alerter-script\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-system-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-cni-bin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-kubelet\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-env-overrides\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d7gf\" (UniqueName: \"kubernetes.io/projected/24b7b4e3-bc45-466d-905f-6e694553f73c-kube-api-access-9d7gf\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-registration-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-kubelet-config\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.416229 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413566 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-systemd\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxhzf\" (UniqueName: \"kubernetes.io/projected/ab101994-6344-4297-ad93-deb48776b8e8-kube-api-access-dxhzf\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413641 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-cni-bin\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-cnibin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413686 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-cnibin\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-conf-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-systemd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-config\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbcd6\" (UniqueName: \"kubernetes.io/projected/6be354b0-ab22-49c7-826b-60a7886c2f21-kube-api-access-cbcd6\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413821 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.416935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zltdq\" (UniqueName: \"kubernetes.io/projected/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kube-api-access-zltdq\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysconfig\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413872 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-system-cni-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413953 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-host-var-lib-kubelet\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.413943 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-script-lib\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.414303 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab101994-6344-4297-ad93-deb48776b8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.414730 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b7b4e3-bc45-466d-905f-6e694553f73c-iptables-alerter-script\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.416441 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be95066e-d25d-4913-bdfe-789f97dd3b31-multus-conf-dir\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.416519 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02816b2d-54ac-4243-b607-535be7d41098-run-systemd\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.417017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab101994-6344-4297-ad93-deb48776b8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.417736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.417093 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02816b2d-54ac-4243-b607-535be7d41098-ovnkube-config\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.418265 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.418247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02816b2d-54ac-4243-b607-535be7d41098-ovn-node-metrics-cert\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.418714 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.418691 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b8eab1-91fd-49bf-a7a7-4fc0e957e76c-agent-certs\") pod \"konnectivity-agent-mmwfv\" (UID: \"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c\") " pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.424355 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.424327 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crs92\" (UniqueName: \"kubernetes.io/projected/9b02124c-6de0-4c0e-83ed-914d2142429c-kube-api-access-crs92\") pod \"node-ca-t8q4v\" (UID: \"9b02124c-6de0-4c0e-83ed-914d2142429c\") " pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.424552 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.424532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2crrf\" (UniqueName: \"kubernetes.io/projected/02816b2d-54ac-4243-b607-535be7d41098-kube-api-access-2crrf\") pod \"ovnkube-node-w4qpz\" (UID: \"02816b2d-54ac-4243-b607-535be7d41098\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.424742 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.424723 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkb6j\" (UniqueName: \"kubernetes.io/projected/be95066e-d25d-4913-bdfe-789f97dd3b31-kube-api-access-lkb6j\") pod \"multus-hbpq8\" (UID: \"be95066e-d25d-4913-bdfe-789f97dd3b31\") " pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.430298 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.430268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d7gf\" (UniqueName: \"kubernetes.io/projected/24b7b4e3-bc45-466d-905f-6e694553f73c-kube-api-access-9d7gf\") pod \"iptables-alerter-rwn98\" (UID: \"24b7b4e3-bc45-466d-905f-6e694553f73c\") " pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.430625 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.430603 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxhzf\" (UniqueName: \"kubernetes.io/projected/ab101994-6344-4297-ad93-deb48776b8e8-kube-api-access-dxhzf\") pod \"multus-additional-cni-plugins-nxs4p\" (UID: \"ab101994-6344-4297-ad93-deb48776b8e8\") " pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.432648 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.432621 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbcd6\" (UniqueName: \"kubernetes.io/projected/6be354b0-ab22-49c7-826b-60a7886c2f21-kube-api-access-cbcd6\") pod \"node-resolver-sphrk\" (UID: \"6be354b0-ab22-49c7-826b-60a7886c2f21\") " pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.432969 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.432952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nn8m\" (UniqueName: \"kubernetes.io/projected/639ebdf2-6bbe-42a5-8ade-322a278cad2d-kube-api-access-7nn8m\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.475154 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.475091 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:38:48 +0000 UTC" deadline="2027-12-16 12:47:36.357111965 +0000 UTC" Jul 07 15:43:49.475154 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.475144 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12645h3m46.881972073s" Jul 07 15:43:49.515217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-host\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-device-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-modprobe-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-conf\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-tmp\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-host\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515326 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-device-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-etc-tuned\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-modprobe-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515405 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515478 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-conf\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515490 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-registration-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.515516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-kubelet-config\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-systemd\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515535 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-registration-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515567 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-kubelet-config\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515611 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-systemd\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zltdq\" (UniqueName: \"kubernetes.io/projected/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kube-api-access-zltdq\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.515651 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysconfig\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515696 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-kubernetes\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.515718 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:50.01569672 +0000 UTC m=+3.246762989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysconfig\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515745 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-kubernetes\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516196 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-run\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-lib-modules\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-dbus\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515853 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-etc-sysctl-d\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-var-lib-kubelet\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515877 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-run\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-socket-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515927 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-sys-fs\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-dbus\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdk6t\" (UniqueName: \"kubernetes.io/projected/2bafa631-7145-4626-92c2-e83051b58768-kube-api-access-qdk6t\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-lib-modules\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.515976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.516005 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-var-lib-kubelet\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.516025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-sys\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.516053 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-socket-dir\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.516948 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.516109 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bafa631-7145-4626-92c2-e83051b58768-sys\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.517730 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.516208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-sys-fs\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.517854 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.517767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-etc-tuned\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.518115 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.518097 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bafa631-7145-4626-92c2-e83051b58768-tmp\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.527815 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.527792 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:43:49.527815 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.527815 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:43:49.527953 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.527827 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:49.527953 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.527902 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:50.027888504 +0000 UTC m=+3.258954762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:49.532172 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.532108 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zltdq\" (UniqueName: \"kubernetes.io/projected/fad19e20-eaf9-49c6-9cc5-3ec484d9413f-kube-api-access-zltdq\") pod \"aws-ebs-csi-driver-node-zgkl9\" (UID: \"fad19e20-eaf9-49c6-9cc5-3ec484d9413f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.532348 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.532332 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdk6t\" (UniqueName: \"kubernetes.io/projected/2bafa631-7145-4626-92c2-e83051b58768-kube-api-access-qdk6t\") pod \"tuned-5nk8h\" (UID: \"2bafa631-7145-4626-92c2-e83051b58768\") " pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.597659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.597618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sphrk" Jul 07 15:43:49.607222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.607201 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rwn98" Jul 07 15:43:49.608707 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.608681 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be354b0_ab22_49c7_826b_60a7886c2f21.slice/crio-2b58aeb653e0913809ea6802b8aaaca6ab63f290140bd327a1c1a939adde18e3 WatchSource:0}: Error finding container 2b58aeb653e0913809ea6802b8aaaca6ab63f290140bd327a1c1a939adde18e3: Status 404 returned error can't find the container with id 2b58aeb653e0913809ea6802b8aaaca6ab63f290140bd327a1c1a939adde18e3 Jul 07 15:43:49.614983 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.614960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:43:49.616037 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.616013 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b7b4e3_bc45_466d_905f_6e694553f73c.slice/crio-765ff1087f121929beb53520335939995b74d55bfc3f1ff5891dcf68939fb3fc WatchSource:0}: Error finding container 765ff1087f121929beb53520335939995b74d55bfc3f1ff5891dcf68939fb3fc: Status 404 returned error can't find the container with id 765ff1087f121929beb53520335939995b74d55bfc3f1ff5891dcf68939fb3fc Jul 07 15:43:49.620346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.620325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8q4v" Jul 07 15:43:49.624782 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.624758 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b8eab1_91fd_49bf_a7a7_4fc0e957e76c.slice/crio-fe45cdb528e81dbb291ea1eb02615c7e6af3246b45e3f568bbcda40e9acb16e8 WatchSource:0}: Error finding container fe45cdb528e81dbb291ea1eb02615c7e6af3246b45e3f568bbcda40e9acb16e8: Status 404 returned error can't find the container with id fe45cdb528e81dbb291ea1eb02615c7e6af3246b45e3f568bbcda40e9acb16e8 Jul 07 15:43:49.625416 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.625379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:43:49.629641 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.629619 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b02124c_6de0_4c0e_83ed_914d2142429c.slice/crio-4ebbc1ebdfdd70566a1aea952e43b2913a2de9e5a9231226cd71949cc0586ee7 WatchSource:0}: Error finding container 4ebbc1ebdfdd70566a1aea952e43b2913a2de9e5a9231226cd71949cc0586ee7: Status 404 returned error can't find the container with id 4ebbc1ebdfdd70566a1aea952e43b2913a2de9e5a9231226cd71949cc0586ee7 Jul 07 15:43:49.631394 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.631377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" Jul 07 15:43:49.634408 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.634306 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02816b2d_54ac_4243_b607_535be7d41098.slice/crio-942c0e4304c38aed8744a290eaa7071f29c6ea08b6b24fa03c82028e4177d152 WatchSource:0}: Error finding container 942c0e4304c38aed8744a290eaa7071f29c6ea08b6b24fa03c82028e4177d152: Status 404 returned error can't find the container with id 942c0e4304c38aed8744a290eaa7071f29c6ea08b6b24fa03c82028e4177d152 Jul 07 15:43:49.637311 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.637295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hbpq8" Jul 07 15:43:49.640284 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.640264 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab101994_6344_4297_ad93_deb48776b8e8.slice/crio-a59ac8d804b88d552e87aebe346662eec92b4ad76b10195e7f6ac961fed5d9ef WatchSource:0}: Error finding container a59ac8d804b88d552e87aebe346662eec92b4ad76b10195e7f6ac961fed5d9ef: Status 404 returned error can't find the container with id a59ac8d804b88d552e87aebe346662eec92b4ad76b10195e7f6ac961fed5d9ef Jul 07 15:43:49.644187 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.643552 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" Jul 07 15:43:49.648650 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.648627 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" Jul 07 15:43:49.651455 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.651425 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe95066e_d25d_4913_bdfe_789f97dd3b31.slice/crio-c2fe359c83891abc83886397cc5d268664e907ce58dbb88de5a1874c91d1928b WatchSource:0}: Error finding container c2fe359c83891abc83886397cc5d268664e907ce58dbb88de5a1874c91d1928b: Status 404 returned error can't find the container with id c2fe359c83891abc83886397cc5d268664e907ce58dbb88de5a1874c91d1928b Jul 07 15:43:49.654020 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.653970 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad19e20_eaf9_49c6_9cc5_3ec484d9413f.slice/crio-e5ca0331dc52ad784ba94890b358feab79ecc4367a4a26ec26dbd1b49f5b45f8 WatchSource:0}: Error finding container e5ca0331dc52ad784ba94890b358feab79ecc4367a4a26ec26dbd1b49f5b45f8: Status 404 returned error can't find the container with id e5ca0331dc52ad784ba94890b358feab79ecc4367a4a26ec26dbd1b49f5b45f8 Jul 07 15:43:49.659617 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:43:49.659589 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bafa631_7145_4626_92c2_e83051b58768.slice/crio-5db12d679bc9e46c0eab84d7253f8f9a506228b93c2c0431ab6208d81a0cca26 WatchSource:0}: Error finding container 5db12d679bc9e46c0eab84d7253f8f9a506228b93c2c0431ab6208d81a0cca26: Status 404 returned error can't find the container with id 5db12d679bc9e46c0eab84d7253f8f9a506228b93c2c0431ab6208d81a0cca26 Jul 07 15:43:49.799958 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.799869 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:43:49.918818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.918771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:49.919006 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.918947 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:49.919069 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:49.919020 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:43:50.91900025 +0000 UTC m=+4.150066507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:49.930033 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.929961 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mmwfv" event={"ID":"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c","Type":"ContainerStarted","Data":"fe45cdb528e81dbb291ea1eb02615c7e6af3246b45e3f568bbcda40e9acb16e8"} Jul 07 15:43:49.932891 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.932853 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rwn98" event={"ID":"24b7b4e3-bc45-466d-905f-6e694553f73c","Type":"ContainerStarted","Data":"765ff1087f121929beb53520335939995b74d55bfc3f1ff5891dcf68939fb3fc"} Jul 07 15:43:49.934415 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.934369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sphrk" event={"ID":"6be354b0-ab22-49c7-826b-60a7886c2f21","Type":"ContainerStarted","Data":"2b58aeb653e0913809ea6802b8aaaca6ab63f290140bd327a1c1a939adde18e3"} Jul 07 15:43:49.936333 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.936302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" event={"ID":"db9c52c440a1356e3cef0bcdda630a66","Type":"ContainerStarted","Data":"c826bb5d39829e2f5bb3610dd112811befb22fbfe64bc5f248fb5924c3cbe98c"} Jul 07 15:43:49.937893 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.937868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" event={"ID":"ff025dfd860c0fee39bf6f7d27498ac0","Type":"ContainerStarted","Data":"a08ba2c5c0be3b78f38dc141009cecfc7a08edc5bc5f9d38edb6bfdc2750df42"} Jul 07 15:43:49.939424 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.939402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" event={"ID":"2bafa631-7145-4626-92c2-e83051b58768","Type":"ContainerStarted","Data":"5db12d679bc9e46c0eab84d7253f8f9a506228b93c2c0431ab6208d81a0cca26"} Jul 07 15:43:49.940767 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.940739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" event={"ID":"fad19e20-eaf9-49c6-9cc5-3ec484d9413f","Type":"ContainerStarted","Data":"e5ca0331dc52ad784ba94890b358feab79ecc4367a4a26ec26dbd1b49f5b45f8"} Jul 07 15:43:49.943073 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.942921 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hbpq8" event={"ID":"be95066e-d25d-4913-bdfe-789f97dd3b31","Type":"ContainerStarted","Data":"c2fe359c83891abc83886397cc5d268664e907ce58dbb88de5a1874c91d1928b"} Jul 07 15:43:49.947181 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.947149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerStarted","Data":"a59ac8d804b88d552e87aebe346662eec92b4ad76b10195e7f6ac961fed5d9ef"} Jul 07 15:43:49.948640 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.948614 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"942c0e4304c38aed8744a290eaa7071f29c6ea08b6b24fa03c82028e4177d152"} Jul 07 15:43:49.949868 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:49.949840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8q4v" event={"ID":"9b02124c-6de0-4c0e-83ed-914d2142429c","Type":"ContainerStarted","Data":"4ebbc1ebdfdd70566a1aea952e43b2913a2de9e5a9231226cd71949cc0586ee7"} Jul 07 15:43:50.019713 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.019670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:50.019896 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.019870 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:50.019984 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.019956 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:51.019932902 +0000 UTC m=+4.250999176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:50.120677 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.120580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:50.120854 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.120794 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:43:50.120854 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.120818 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:43:50.120854 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.120831 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:50.121017 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.120894 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:51.120874508 +0000 UTC m=+4.351940767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:50.476098 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.475921 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:38:48 +0000 UTC" deadline="2028-01-07 15:24:46.814128265 +0000 UTC" Jul 07 15:43:50.476098 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.475989 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13175h40m56.338143979s" Jul 07 15:43:50.920813 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.920728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:50.920998 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.920875 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:50.921350 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.921328 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:50.921478 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.921430 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:50.921546 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.921514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:50.921607 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.921584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:43:50.928089 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:50.928057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:50.928317 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.928297 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:50.928378 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:50.928368 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:43:52.928346407 +0000 UTC m=+6.159412665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:51.029177 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:51.029138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:51.029355 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.029294 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:51.029432 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.029363 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:53.029344089 +0000 UTC m=+6.260410348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:51.130695 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:51.130005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:51.130695 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.130184 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:43:51.130695 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.130202 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:43:51.130695 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.130216 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:51.130695 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:51.130278 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:53.130257984 +0000 UTC m=+6.361324243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:52.920761 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:52.920722 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:52.921284 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:52.920885 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:52.921351 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:52.921316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:52.921448 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:52.921418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:43:52.921521 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:52.921497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:52.921624 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:52.921587 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:52.954910 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:52.954767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:52.954910 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:52.954902 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:52.955163 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:52.954968 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:43:56.95494866 +0000 UTC m=+10.186014923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:53.055508 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:53.055469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:53.055693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.055621 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:53.055693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.055687 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:57.055666198 +0000 UTC m=+10.286732474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:53.156490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:53.156329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:53.156697 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.156541 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:43:53.156697 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.156565 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:43:53.156697 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.156580 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:53.156697 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:53.156646 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:43:57.156624552 +0000 UTC m=+10.387690810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:54.920985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:54.920938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:54.920985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:54.920980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:54.921663 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:54.921091 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:43:54.921663 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:54.921195 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:54.921663 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:54.921249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:54.921663 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:54.921324 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:56.920023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:56.919990 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:56.920493 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:56.920106 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:56.920493 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:56.920182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:56.920493 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:56.920267 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:56.920493 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:56.920316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:56.920493 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:56.920400 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:43:56.991323 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:56.991279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:56.991510 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:56.991450 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:56.991589 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:56.991523 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:44:04.991502234 +0000 UTC m=+18.222568491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:43:57.092021 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:57.091978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:57.092191 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.092141 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:57.092239 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.092210 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:05.092192551 +0000 UTC m=+18.323258805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:43:57.193288 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:57.192612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:57.193288 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.192804 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:43:57.193288 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.192826 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:43:57.193288 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.192839 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:57.193288 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:57.192896 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:05.192878448 +0000 UTC m=+18.423944715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:43:58.919985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:58.919950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:43:58.920451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:58.919993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:43:58.920451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:43:58.919998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:43:58.920451 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:58.920130 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:43:58.920451 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:58.920328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:43:58.920451 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:43:58.920418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:00.920397 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:00.920350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:00.920786 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:00.920364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:00.920786 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:00.920495 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:00.920786 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:00.920555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:00.920786 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:00.920659 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:00.920786 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:00.920731 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:02.920976 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:02.920944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:02.921463 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:02.920944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:02.921463 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:02.921068 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:02.921463 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:02.920948 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:02.921463 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:02.921152 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:02.921463 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:02.921231 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:04.920443 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:04.920407 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:04.920443 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:04.920431 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:04.920991 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:04.920461 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:04.920991 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:04.920583 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:04.920991 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:04.920657 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:04.920991 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:04.920773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:05.049434 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:05.049398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:05.049592 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.049564 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:05.049665 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.049654 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:44:21.04963899 +0000 UTC m=+34.280705243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:05.150727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:05.150687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:05.150893 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.150813 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:05.150893 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.150865 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:21.150851639 +0000 UTC m=+34.381917892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:05.251681 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:05.251599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:05.251820 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.251750 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:05.251820 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.251773 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:05.251820 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.251782 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:05.251932 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:05.251830 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:21.251816922 +0000 UTC m=+34.482883176 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:06.920723 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:06.920690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:06.921135 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:06.920757 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:06.921135 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:06.920871 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:06.921135 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:06.920868 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:06.921135 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:06.921005 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:06.921135 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:06.921080 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:08.920658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:08.920626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:08.921076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:08.920623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:08.921076 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:08.920756 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:08.921076 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:08.920845 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:08.921076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:08.920629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:08.921076 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:08.920950 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:10.005526 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.005478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" event={"ID":"db9c52c440a1356e3cef0bcdda630a66","Type":"ContainerStarted","Data":"cc6f279b3eb0ee2d3dfa3c03e542901cde85a3212e767be0a7979e891b15d746"} Jul 07 15:44:10.011680 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.011650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" event={"ID":"2bafa631-7145-4626-92c2-e83051b58768","Type":"ContainerStarted","Data":"673430fa0ea6e270a79459ebef4abf02e61064d3a1bbb4fa65d11b7d9b5ef1b7"} Jul 07 15:44:10.014340 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.014305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hbpq8" event={"ID":"be95066e-d25d-4913-bdfe-789f97dd3b31","Type":"ContainerStarted","Data":"fe23b7becdf34f0fa9452fe426ce5092b9593d29df32a6bd5b60e5ea4dab4a7f"} Jul 07 15:44:10.024687 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.024608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.ec2.internal" podStartSLOduration=22.024588661 podStartE2EDuration="22.024588661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:10.02417795 +0000 UTC m=+23.255244261" watchObservedRunningTime="2026-07-07 15:44:10.024588661 +0000 UTC m=+23.255654937" Jul 07 15:44:10.045915 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.045778 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5nk8h" podStartSLOduration=2.007128747 podStartE2EDuration="22.045755716s" totalImagesPullingTime="20.038626969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:10.045103578 +0000 UTC m=+23.276169872" watchObservedRunningTime="2026-07-07 15:44:10.045755716 +0000 UTC m=+23.276821992" Jul 07 15:44:10.065591 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.065545 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hbpq8" podStartSLOduration=1.982358437 podStartE2EDuration="22.065529508s" totalImagesPullingTime="20.083171071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:10.065261869 +0000 UTC m=+23.296328145" watchObservedRunningTime="2026-07-07 15:44:10.065529508 +0000 UTC m=+23.296595787" Jul 07 15:44:10.920326 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.920147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:10.920712 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.920148 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:10.920712 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:10.920418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:10.920712 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:10.920147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:10.920712 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:10.920470 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:10.920712 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:10.920567 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:11.017555 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.017466 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" event={"ID":"fad19e20-eaf9-49c6-9cc5-3ec484d9413f","Type":"ContainerStarted","Data":"b79459d4cedeff43ab711c1fe8a1649948021d0736b14f2803eceac8bb495cf2"} Jul 07 15:44:11.018980 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.018958 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="51f31cf2bfaa662017f4870fb9e0e3b476eed8375b8bb81e1ab064a000ef6dee" exitCode=0 Jul 07 15:44:11.019075 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.019032 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"51f31cf2bfaa662017f4870fb9e0e3b476eed8375b8bb81e1ab064a000ef6dee"} Jul 07 15:44:11.021961 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.021941 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:44:11.022282 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022263 2562 generic.go:356] "Generic (PLEG): container finished" podID="02816b2d-54ac-4243-b607-535be7d41098" containerID="2bf86141c8fd269d7958b4abc2a211ed6c8c8e2dc0e0d22ee76fb08e351b3c9e" exitCode=1 Jul 07 15:44:11.022353 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"7729d7de9e9ea7e40fac0e43930386768a17e555c4acc30c8d3260d7fce80409"} Jul 07 15:44:11.022439 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022416 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"b8f0d1f984548c6a1d746079fb4dc0b635e301aaf1df15d77077a50f41db0593"} Jul 07 15:44:11.022497 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"d56e39c1e4070831dfd684c98c4ce78c7cef4bc37d7a8d7907e334ad2d4c2bbb"} Jul 07 15:44:11.022542 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"9b54d27c62bcd3293666e2c36a577b9f703ed2ae4642e4eb6a4e84fdd5551eaa"} Jul 07 15:44:11.022542 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerDied","Data":"2bf86141c8fd269d7958b4abc2a211ed6c8c8e2dc0e0d22ee76fb08e351b3c9e"} Jul 07 15:44:11.022636 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.022571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"7dfba068b91eddcbce88f8ab9c39203bc2c031003234088464415a912afddb54"} Jul 07 15:44:11.023837 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.023816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8q4v" event={"ID":"9b02124c-6de0-4c0e-83ed-914d2142429c","Type":"ContainerStarted","Data":"0daf0ad9a4017b087e6c8d40178f43fb3978eea4cf5ee59d4e4d2ec998ad8939"} Jul 07 15:44:11.025103 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.025080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mmwfv" event={"ID":"33b8eab1-91fd-49bf-a7a7-4fc0e957e76c","Type":"ContainerStarted","Data":"61d501d86bfeae0fd84ebb2dfe99397ce3b91676c736c8c2501824426de4f176"} Jul 07 15:44:11.026469 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.026447 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rwn98" event={"ID":"24b7b4e3-bc45-466d-905f-6e694553f73c","Type":"ContainerStarted","Data":"5d193bef14c21f89aa0e944f16aa6fc95ec291f644ce1b81eff42c320f5a5979"} Jul 07 15:44:11.027720 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.027689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sphrk" event={"ID":"6be354b0-ab22-49c7-826b-60a7886c2f21","Type":"ContainerStarted","Data":"2f546314b707151079fb2a24c35073eb8197febcdfb12bbbfa12b7d8cb013e93"} Jul 07 15:44:11.028895 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.028876 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff025dfd860c0fee39bf6f7d27498ac0" containerID="040eef8976c2ee86f31b3863bd00db8388ccab009f674271bce8ffb7bc5c2b79" exitCode=0 Jul 07 15:44:11.028986 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.028927 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" event={"ID":"ff025dfd860c0fee39bf6f7d27498ac0","Type":"ContainerDied","Data":"040eef8976c2ee86f31b3863bd00db8388ccab009f674271bce8ffb7bc5c2b79"} Jul 07 15:44:11.062396 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.062308 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sphrk" podStartSLOduration=2.974488819 podStartE2EDuration="23.062290578s" totalImagesPullingTime="20.087801759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:11.061885955 +0000 UTC m=+24.292952243" watchObservedRunningTime="2026-07-07 15:44:11.062290578 +0000 UTC m=+24.293356854" Jul 07 15:44:11.078551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.078489 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t8q4v" podStartSLOduration=11.448565475 podStartE2EDuration="23.078474627s" totalImagesPullingTime="11.629909152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:11.077817657 +0000 UTC m=+24.308883953" watchObservedRunningTime="2026-07-07 15:44:11.078474627 +0000 UTC m=+24.309540903" Jul 07 15:44:11.118970 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.118927 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rwn98" podStartSLOduration=3.037950836 podStartE2EDuration="23.118909897s" totalImagesPullingTime="20.080959061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:11.118370235 +0000 UTC m=+24.349436512" watchObservedRunningTime="2026-07-07 15:44:11.118909897 +0000 UTC m=+24.349976173" Jul 07 15:44:11.133756 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:11.133700 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mmwfv" podStartSLOduration=3.060624704 podStartE2EDuration="23.133682315s" totalImagesPullingTime="20.073057611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:11.133343603 +0000 UTC m=+24.364409912" watchObservedRunningTime="2026-07-07 15:44:11.133682315 +0000 UTC m=+24.364748625" Jul 07 15:44:12.033443 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.033412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" event={"ID":"ff025dfd860c0fee39bf6f7d27498ac0","Type":"ContainerStarted","Data":"40bbb7e7371e40a243edc464b7b99686ec6ff1d039d541397cd2bba20aa4026f"} Jul 07 15:44:12.075760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.075736 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:44:12.780300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.778956 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:44:12.780300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.779429 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:44:12.797427 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.797145 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.ec2.internal" podStartSLOduration=24.797123944 podStartE2EDuration="24.797123944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:12.053530096 +0000 UTC m=+25.284596384" watchObservedRunningTime="2026-07-07 15:44:12.797123944 +0000 UTC m=+26.028190220" Jul 07 15:44:12.882598 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.882502 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:44:12.075754911Z","UUID":"671a5bb3-ed74-402b-922c-790c16c605c1","Handler":null,"Name":"","Endpoint":""} Jul 07 15:44:12.884615 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.884526 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:44:12.884615 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.884573 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:44:12.920159 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.920097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:12.920273 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.920194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:12.920273 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:12.920200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:12.920273 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:12.920219 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:12.920423 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:12.920301 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:12.920423 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:12.920370 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:13.038187 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:13.038124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" event={"ID":"fad19e20-eaf9-49c6-9cc5-3ec484d9413f","Type":"ContainerStarted","Data":"6ca9f6b39d9d70affdb7215fb67618b92e1144e6d4000638bd752b486fe79d6a"} Jul 07 15:44:13.041994 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:13.041968 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:44:13.042494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:13.042368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"d081db5a078c715d2f2d622659c6988e290520a71b75962f5e84dbdaf57d51a0"} Jul 07 15:44:13.043020 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:13.042979 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:44:13.043390 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:13.043357 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mmwfv" Jul 07 15:44:14.046022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:14.045989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" event={"ID":"fad19e20-eaf9-49c6-9cc5-3ec484d9413f","Type":"ContainerStarted","Data":"42b9fbdccbe3fa7f122a513b340048ad53efde74aea79204155704884ab3cb39"} Jul 07 15:44:14.070079 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:14.070026 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zgkl9" podStartSLOduration=3.310270059 podStartE2EDuration="26.070010287s" totalImagesPullingTime="22.759740228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:14.068254323 +0000 UTC m=+27.299320597" watchObservedRunningTime="2026-07-07 15:44:14.070010287 +0000 UTC m=+27.301076562" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:14.920189 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:14.920238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:14.920550 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:14.920576 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:14.920238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:14.921465 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:14.920646 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:15.052826 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:15.052748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:44:15.053647 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:15.053210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"dd77811f62048a3b39f53418a98447c3e00a16fb63f18b93ec10f7c3175d557c"} Jul 07 15:44:15.053747 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:15.053728 2562 scope.go:117] "RemoveContainer" containerID="2bf86141c8fd269d7958b4abc2a211ed6c8c8e2dc0e0d22ee76fb08e351b3c9e" Jul 07 15:44:16.062417 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.062372 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:44:16.063097 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.062785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" event={"ID":"02816b2d-54ac-4243-b607-535be7d41098","Type":"ContainerStarted","Data":"d90ddd54be7fb472604f3ba891cc1285d5e4eebd2e60f9ac7222c3f48ee89e70"} Jul 07 15:44:16.063097 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.062907 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:44:16.063220 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.063210 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:16.063296 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.063282 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:16.084045 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.084005 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:16.086583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.086560 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:16.142168 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.142098 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" podStartSLOduration=7.831133478 podStartE2EDuration="28.142075815s" totalImagesPullingTime="20.310942337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:16.104552096 +0000 UTC m=+29.335618374" watchObservedRunningTime="2026-07-07 15:44:16.142075815 +0000 UTC m=+29.373142091" Jul 07 15:44:16.920849 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.920660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:16.921009 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.920681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:16.921009 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:16.920923 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:16.921009 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:16.920681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:16.921138 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:16.921024 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:16.921138 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:16.921060 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:17.065158 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.065131 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:44:17.103676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.103646 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9mlh8"] Jul 07 15:44:17.103855 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.103752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:17.103855 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:17.103844 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:17.106299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.106276 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbdb7"] Jul 07 15:44:17.106443 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.106357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:17.106488 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:17.106455 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:17.110322 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.110299 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-42jm9"] Jul 07 15:44:17.110447 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:17.110406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:17.110544 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:17.110525 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:18.068511 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.068476 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="f877f421c49bbb734030631d3706a357f111e66dedaa7e1762d93715b52d18fd" exitCode=0 Jul 07 15:44:18.068973 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.068562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"f877f421c49bbb734030631d3706a357f111e66dedaa7e1762d93715b52d18fd"} Jul 07 15:44:18.068973 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.068646 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:44:18.920048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.920017 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:18.920576 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.920058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:18.920576 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.920019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:18.920576 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:18.920143 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:18.920576 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:18.920240 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:18.920576 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:18.920338 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:18.963041 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:18.963009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:19.091587 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:19.091547 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" podUID="02816b2d-54ac-4243-b607-535be7d41098" containerName="ovnkube-controller" probeResult="failure" output="" Jul 07 15:44:20.075309 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:20.075272 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="026218ab8bda1b789c6487c39f3ca049d00872fafb8cfeef3ad10931396ac13c" exitCode=0 Jul 07 15:44:20.075490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:20.075328 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"026218ab8bda1b789c6487c39f3ca049d00872fafb8cfeef3ad10931396ac13c"} Jul 07 15:44:20.920934 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:20.920901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:20.921311 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:20.920901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:20.921311 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:20.921015 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9mlh8" podUID="de821d61-5eb5-47bd-a9ce-24fe4b4871f6" Jul 07 15:44:20.921311 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:20.920901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:20.921311 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:20.921097 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42jm9" podUID="639ebdf2-6bbe-42a5-8ade-322a278cad2d" Jul 07 15:44:20.921311 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:20.921167 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbdb7" podUID="e10b46df-00c3-4213-97ca-84bd168cd7b3" Jul 07 15:44:21.064880 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:21.064847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:21.065053 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.065017 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:21.065112 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.065094 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:44:53.065072279 +0000 UTC m=+66.296138629 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:44:21.165815 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:21.165781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:21.165994 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.165904 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:21.165994 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.165957 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret podName:de821d61-5eb5-47bd-a9ce-24fe4b4871f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:53.165941986 +0000 UTC m=+66.397008240 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret") pod "global-pull-secret-syncer-9mlh8" (UID: "de821d61-5eb5-47bd-a9ce-24fe4b4871f6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:44:21.266341 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:21.266308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:21.266508 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.266480 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:44:21.266508 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.266501 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:44:21.266579 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.266511 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sszkf for pod openshift-network-diagnostics/network-check-target-qbdb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:21.266579 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:21.266559 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf podName:e10b46df-00c3-4213-97ca-84bd168cd7b3 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:53.266546455 +0000 UTC m=+66.497612708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sszkf" (UniqueName: "kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf") pod "network-check-target-qbdb7" (UID: "e10b46df-00c3-4213-97ca-84bd168cd7b3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:44:22.082606 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.082328 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="c1d6e0d127b7fb1b7d6556fb1595465bc7f7773013196c2ac49e8d3126ec870f" exitCode=0 Jul 07 15:44:22.082606 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.082409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"c1d6e0d127b7fb1b7d6556fb1595465bc7f7773013196c2ac49e8d3126ec870f"} Jul 07 15:44:22.566513 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.566486 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-42.ec2.internal" event="NodeReady" Jul 07 15:44:22.566648 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.566595 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:44:22.608540 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.608462 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:44:22.610526 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.610510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.614048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.614019 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 15:44:22.614147 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.614100 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 15:44:22.614288 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.614273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 15:44:22.614419 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.614402 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fglhg\"" Jul 07 15:44:22.622238 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.622217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 15:44:22.631050 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.631024 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2kbdn"] Jul 07 15:44:22.633291 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.633276 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.637346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.637309 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:44:22.637346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.637331 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:44:22.637346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.637516 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmh7"] Jul 07 15:44:22.637757 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.637615 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j6trp\"" Jul 07 15:44:22.639765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.639693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:44:22.639903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.639887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.643153 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.642807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:44:22.643153 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.642844 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vwnd2\"" Jul 07 15:44:22.643153 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.642985 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:44:22.643153 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.643128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:44:22.650230 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.650206 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2kbdn"] Jul 07 15:44:22.654335 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.654316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmh7"] Jul 07 15:44:22.778455 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.778604 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/401d0f3f-db12-451d-8452-d3348ccd8de9-config-volume\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.778604 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd6h4\" (UniqueName: \"kubernetes.io/projected/401d0f3f-db12-451d-8452-d3348ccd8de9-kube-api-access-dd6h4\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.778604 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24bbd909-d7bd-453e-83eb-6c1d5120b132-cert\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401d0f3f-db12-451d-8452-d3348ccd8de9-tmp-dir\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.778739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.779005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzw8w\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.779005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.779005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbzzp\" (UniqueName: \"kubernetes.io/projected/24bbd909-d7bd-453e-83eb-6c1d5120b132-kube-api-access-gbzzp\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.779005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.778930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.807469 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.807432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:22.880203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbzzp\" (UniqueName: \"kubernetes.io/projected/24bbd909-d7bd-453e-83eb-6c1d5120b132-kube-api-access-gbzzp\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.880203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880434 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880434 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/401d0f3f-db12-451d-8452-d3348ccd8de9-config-volume\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.880434 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd6h4\" (UniqueName: \"kubernetes.io/projected/401d0f3f-db12-451d-8452-d3348ccd8de9-kube-api-access-dd6h4\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.880551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.880551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880710 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24bbd909-d7bd-453e-83eb-6c1d5120b132-cert\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.880710 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401d0f3f-db12-451d-8452-d3348ccd8de9-tmp-dir\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.880710 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:22.880673 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:22.880710 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880871 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzw8w\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.880927 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880880 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/401d0f3f-db12-451d-8452-d3348ccd8de9-config-volume\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.880927 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.880901 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.881015 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:22.880689 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:22.881015 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:22.880945 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:22.881015 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:22.880737 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:23.380718605 +0000 UTC m=+36.611784878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:22.881015 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:22.881012 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:23.38099394 +0000 UTC m=+36.612060206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:22.881166 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.881051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.881166 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.881105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401d0f3f-db12-451d-8452-d3348ccd8de9-tmp-dir\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.881356 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.881333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.881903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.881876 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.885718 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.885696 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.885818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.885744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.885818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.885793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24bbd909-d7bd-453e-83eb-6c1d5120b132-cert\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.891312 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.891292 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbzzp\" (UniqueName: \"kubernetes.io/projected/24bbd909-d7bd-453e-83eb-6c1d5120b132-kube-api-access-gbzzp\") pod \"ingress-canary-5rmh7\" (UID: \"24bbd909-d7bd-453e-83eb-6c1d5120b132\") " pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.891900 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.891854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.892027 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.892008 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzw8w\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:22.893024 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.892984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd6h4\" (UniqueName: \"kubernetes.io/projected/401d0f3f-db12-451d-8452-d3348ccd8de9-kube-api-access-dd6h4\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:22.920438 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.920408 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:22.920586 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.920441 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:22.920586 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.920447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:22.923873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.923738 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:44:22.923873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.923774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:44:22.923873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.923797 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wcvdn\"" Jul 07 15:44:22.923873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.923797 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ctzps\"" Jul 07 15:44:22.924144 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.923738 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:44:22.925203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.925177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:44:22.949124 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.949105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5rmh7" Jul 07 15:44:22.980134 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:22.980098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:23.092783 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:23.092754 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5rmh7"] Jul 07 15:44:23.096784 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:23.096755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bbd909_d7bd_453e_83eb_6c1d5120b132.slice/crio-c4569396cf5c34f5fcec9bdd1a175cfad29abe5b3343819725a446c406cf8487 WatchSource:0}: Error finding container c4569396cf5c34f5fcec9bdd1a175cfad29abe5b3343819725a446c406cf8487: Status 404 returned error can't find the container with id c4569396cf5c34f5fcec9bdd1a175cfad29abe5b3343819725a446c406cf8487 Jul 07 15:44:23.385829 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:23.385790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:23.385829 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:23.385827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:23.386030 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:23.385950 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:23.386030 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:23.386005 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:24.385992032 +0000 UTC m=+37.617058289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:23.386101 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:23.385952 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:23.386101 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:23.386046 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:23.386101 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:23.386088 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:24.386076536 +0000 UTC m=+37.617142789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:24.089266 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:24.089228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5rmh7" event={"ID":"24bbd909-d7bd-453e-83eb-6c1d5120b132","Type":"ContainerStarted","Data":"c4569396cf5c34f5fcec9bdd1a175cfad29abe5b3343819725a446c406cf8487"} Jul 07 15:44:24.396005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:24.395889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:24.396005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:24.395938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:24.396693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:24.396017 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:24.396693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:24.396063 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:24.396693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:24.396079 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:24.396693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:24.396089 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:26.396074429 +0000 UTC m=+39.627140685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:24.396693 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:24.396146 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:26.396129278 +0000 UTC m=+39.627195544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:24.561238 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:24.561202 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.096338 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.096264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5rmh7" event={"ID":"24bbd909-d7bd-453e-83eb-6c1d5120b132","Type":"ContainerStarted","Data":"88a35d0ca3259fdd2d9350e91e679991cec2334c4cb0f72c9c6e79c61d385dcb"} Jul 07 15:44:26.136620 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.136555 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5rmh7" podStartSLOduration=1.436280097 podStartE2EDuration="4.13653435s" totalImagesPullingTime="2.700254253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:26.134560904 +0000 UTC m=+39.365627179" watchObservedRunningTime="2026-07-07 15:44:26.13653435 +0000 UTC m=+39.367600626" Jul 07 15:44:26.238015 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.237958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.377483 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.377398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.412658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.412622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:26.412833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.412670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:26.412833 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:26.412786 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:26.412833 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:26.412825 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:26.412967 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:26.412840 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:26.412967 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:26.412858 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:30.41283564 +0000 UTC m=+43.643901892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:26.412967 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:26.412884 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:30.412872641 +0000 UTC m=+43.643938894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:26.512502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.512470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.655426 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.655324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.788346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.788300 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.887746 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.887704 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:26.985638 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:26.985553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:27.216065 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:27.216029 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:27.308332 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:27.308297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:27.644053 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:27.643968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:27.753635 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:27.753603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:27.861527 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:27.861487 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:28.859189 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:28.859157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:28.959723 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:28.959688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:29.063661 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:29.063631 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:29.105903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:29.105875 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="ff5b8b799e68e3bc155d1c8a584c4730f55651d8dd7bfc94f792983b880871cc" exitCode=0 Jul 07 15:44:29.106145 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:29.105924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"ff5b8b799e68e3bc155d1c8a584c4730f55651d8dd7bfc94f792983b880871cc"} Jul 07 15:44:29.828523 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:29.828490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:29.948812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:29.948781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:30.048028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.047997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:30.110676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.110602 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab101994-6344-4297-ad93-deb48776b8e8" containerID="32379e69a98f3d9b380a9e60708ac59e0ce34773934f75837781e338f372540a" exitCode=0 Jul 07 15:44:30.110676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.110659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerDied","Data":"32379e69a98f3d9b380a9e60708ac59e0ce34773934f75837781e338f372540a"} Jul 07 15:44:30.124571 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.124545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:30.445301 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.445217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:30.445301 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:30.445247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:30.445506 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:30.445329 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:30.445506 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:30.445408 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:38.445393694 +0000 UTC m=+51.676459960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:30.445506 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:30.445336 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:30.445506 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:30.445431 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:30.445506 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:30.445471 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:38.445459807 +0000 UTC m=+51.676526060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:31.115760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:31.115725 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" event={"ID":"ab101994-6344-4297-ad93-deb48776b8e8","Type":"ContainerStarted","Data":"9ccb6eb8a4f9c45e5d5476727758b08fae9c58697b2dae8712ef836e06c67707"} Jul 07 15:44:31.145595 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:31.145547 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nxs4p" podStartSLOduration=8.360158499 podStartE2EDuration="43.1455299s" totalImagesPullingTime="34.785371401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:31.144492118 +0000 UTC m=+44.375558411" watchObservedRunningTime="2026-07-07 15:44:31.1455299 +0000 UTC m=+44.376596173" Jul 07 15:44:31.276584 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:31.276550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:31.375718 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:31.375641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:38.509490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:38.509446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:38.509490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:38.509492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:38.509962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:38.509605 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:38.509962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:38.509617 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:38.509962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:38.509617 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:38.509962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:38.509664 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:44:54.509651946 +0000 UTC m=+67.740718200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:38.509962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:38.509688 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:54.509669467 +0000 UTC m=+67.740735721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:40.277548 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.277504 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf"] Jul 07 15:44:40.290562 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.290531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.292170 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.292136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf"] Jul 07 15:44:40.293631 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.293601 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 15:44:40.293765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.293631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.293765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.293607 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 15:44:40.293765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.293637 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.294852 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.294830 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-t6cpc\"" Jul 07 15:44:40.322709 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.322679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.322874 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.322734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z58jq\" (UniqueName: \"kubernetes.io/projected/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-kube-api-access-z58jq\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.322874 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.322781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.386802 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.386746 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps"] Jul 07 15:44:40.405137 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.405107 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv"] Jul 07 15:44:40.405310 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.405249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.408119 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.408096 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 15:44:40.408298 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.408258 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xmr7f\"" Jul 07 15:44:40.408493 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.408431 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.408797 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.408108 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.409890 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.409872 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 15:44:40.423098 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.423215 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z58jq\" (UniqueName: \"kubernetes.io/projected/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-kube-api-access-z58jq\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.423215 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.423215 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-784xn\" (UniqueName: \"kubernetes.io/projected/b2b5f3bd-1764-4092-a798-cf68777254c4-kube-api-access-784xn\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.423357 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b2b5f3bd-1764-4092-a798-cf68777254c4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.423440 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.423637 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.423617 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.425420 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.425402 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm"] Jul 07 15:44:40.425494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.425454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.426537 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.426519 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.428346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.428328 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 15:44:40.428805 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.428512 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.428805 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.428615 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 15:44:40.428926 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.428910 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2w78l\"" Jul 07 15:44:40.429051 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.429031 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.436461 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.436439 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z58jq\" (UniqueName: \"kubernetes.io/projected/cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648-kube-api-access-z58jq\") pod \"kube-storage-version-migrator-operator-846bf668f7-q8xlf\" (UID: \"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.439919 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.439901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.439919 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.439904 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bvhfr"] Jul 07 15:44:40.442913 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.442873 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 15:44:40.442913 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.442905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.443081 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.442918 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-z74qr\"" Jul 07 15:44:40.443081 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.442955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.457750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.457725 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24"] Jul 07 15:44:40.457881 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.457777 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.460925 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.460907 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.461368 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.461286 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 15:44:40.461368 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.461306 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vbf7x\"" Jul 07 15:44:40.461518 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.461403 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.461710 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.461695 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 15:44:40.466835 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.466818 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 15:44:40.480392 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps"] Jul 07 15:44:40.480392 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv"] Jul 07 15:44:40.480541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" Jul 07 15:44:40.480541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480479 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bvhfr"] Jul 07 15:44:40.480541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480493 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24"] Jul 07 15:44:40.480541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480507 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm"] Jul 07 15:44:40.480541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.480533 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr"] Jul 07 15:44:40.483293 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.483276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.483583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.483564 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.487790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.487774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t2shd\"" Jul 07 15:44:40.497670 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.497644 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d96q5"] Jul 07 15:44:40.497783 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.497757 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" Jul 07 15:44:40.501774 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.501737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g6nsc\"" Jul 07 15:44:40.513874 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.513854 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr"] Jul 07 15:44:40.513995 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.513888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d96q5"] Jul 07 15:44:40.513995 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.513908 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-d6c7b58d5-x9h75"] Jul 07 15:44:40.513995 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.513966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.516979 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.516963 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.517067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.516993 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-82wsv\"" Jul 07 15:44:40.517067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.517015 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.517221 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.517197 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 15:44:40.517452 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.517438 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 15:44:40.521954 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.521933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 15:44:40.523994 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.523973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbbl\" (UniqueName: \"kubernetes.io/projected/f8286691-e614-41cc-bffe-79ed7194bfab-kube-api-access-7qbbl\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.524163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-784xn\" (UniqueName: \"kubernetes.io/projected/b2b5f3bd-1764-4092-a798-cf68777254c4-kube-api-access-784xn\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.524163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-config\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.524163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8286691-e614-41cc-bffe-79ed7194bfab-serving-cert\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.524163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqbq\" (UniqueName: \"kubernetes.io/projected/64e727a5-62e8-49d9-8c1b-9ec2638408ac-kube-api-access-zbqbq\") pod \"network-check-source-989f8b8dc-cm8lr\" (UID: \"64e727a5-62e8-49d9-8c1b-9ec2638408ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" Jul 07 15:44:40.524163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-tmp\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppq8r\" (UniqueName: \"kubernetes.io/projected/31228800-77a6-4869-9090-7cbe6fa30887-kube-api-access-ppq8r\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524253 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbr6\" (UniqueName: \"kubernetes.io/projected/397726ac-cf54-475f-aed6-0bc95cf73ad0-kube-api-access-gvbr6\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f702084c-90f6-49db-a2fd-5a5bee9e76be-serving-cert\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lqhv\" (UniqueName: \"kubernetes.io/projected/3670b58c-91c7-48c2-98c3-d3dc3ccd010e-kube-api-access-9lqhv\") pod \"volume-data-source-validator-5d456fdbcd-xdq24\" (UID: \"3670b58c-91c7-48c2-98c3-d3dc3ccd010e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31228800-77a6-4869-9090-7cbe6fa30887-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.524466 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b2b5f3bd-1764-4092-a798-cf68777254c4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv2s8\" (UniqueName: \"kubernetes.io/projected/f702084c-90f6-49db-a2fd-5a5bee9e76be-kube-api-access-gv2s8\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.524593 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.524643 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:41.024624831 +0000 UTC m=+54.255691098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31228800-77a6-4869-9090-7cbe6fa30887-config\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-trusted-ca\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-snapshots\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.524833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.524780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.525346 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.525327 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b2b5f3bd-1764-4092-a798-cf68777254c4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.532259 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.532211 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-784xn\" (UniqueName: \"kubernetes.io/projected/b2b5f3bd-1764-4092-a798-cf68777254c4-kube-api-access-784xn\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:40.533929 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.533908 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d6c7b58d5-x9h75"] Jul 07 15:44:40.534036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.533988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.536770 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.536750 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 15:44:40.537022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.536905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tqbd5\"" Jul 07 15:44:40.537022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.536967 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:44:40.537022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.536905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 15:44:40.537022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.536969 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:44:40.537251 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.537034 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 15:44:40.537251 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.537054 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 15:44:40.600140 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.600100 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" Jul 07 15:44:40.625950 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.625925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.625957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbbl\" (UniqueName: \"kubernetes.io/projected/f8286691-e614-41cc-bffe-79ed7194bfab-kube-api-access-7qbbl\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.625995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-stats-auth\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-config\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-default-certificate\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626073 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.626088 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.626084 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:44:40.626428 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8286691-e614-41cc-bffe-79ed7194bfab-serving-cert\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.626428 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.626428 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.626158 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls podName:397726ac-cf54-475f-aed6-0bc95cf73ad0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:41.126134993 +0000 UTC m=+54.357201254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8hmgm" (UID: "397726ac-cf54-475f-aed6-0bc95cf73ad0") : secret "samples-operator-tls" not found Jul 07 15:44:40.626428 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqbq\" (UniqueName: \"kubernetes.io/projected/64e727a5-62e8-49d9-8c1b-9ec2638408ac-kube-api-access-zbqbq\") pod \"network-check-source-989f8b8dc-cm8lr\" (UID: \"64e727a5-62e8-49d9-8c1b-9ec2638408ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" Jul 07 15:44:40.626428 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqsjn\" (UniqueName: \"kubernetes.io/projected/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-kube-api-access-zqsjn\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-tmp\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppq8r\" (UniqueName: \"kubernetes.io/projected/31228800-77a6-4869-9090-7cbe6fa30887-kube-api-access-ppq8r\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbr6\" (UniqueName: \"kubernetes.io/projected/397726ac-cf54-475f-aed6-0bc95cf73ad0-kube-api-access-gvbr6\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f702084c-90f6-49db-a2fd-5a5bee9e76be-serving-cert\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lqhv\" (UniqueName: \"kubernetes.io/projected/3670b58c-91c7-48c2-98c3-d3dc3ccd010e-kube-api-access-9lqhv\") pod \"volume-data-source-validator-5d456fdbcd-xdq24\" (UID: \"3670b58c-91c7-48c2-98c3-d3dc3ccd010e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" Jul 07 15:44:40.626690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31228800-77a6-4869-9090-7cbe6fa30887-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv2s8\" (UniqueName: \"kubernetes.io/projected/f702084c-90f6-49db-a2fd-5a5bee9e76be-kube-api-access-gv2s8\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31228800-77a6-4869-9090-7cbe6fa30887-config\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-trusted-ca\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-snapshots\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626821 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-config\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626832 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.626870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-tmp\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.627619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f702084c-90f6-49db-a2fd-5a5bee9e76be-snapshots\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.627853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.627699 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31228800-77a6-4869-9090-7cbe6fa30887-config\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.627853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.627775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f702084c-90f6-49db-a2fd-5a5bee9e76be-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.628102 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.627917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8286691-e614-41cc-bffe-79ed7194bfab-trusted-ca\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.628936 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.628895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8286691-e614-41cc-bffe-79ed7194bfab-serving-cert\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.629302 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.629266 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31228800-77a6-4869-9090-7cbe6fa30887-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.629872 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.629854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f702084c-90f6-49db-a2fd-5a5bee9e76be-serving-cert\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.638961 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.638931 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbbl\" (UniqueName: \"kubernetes.io/projected/f8286691-e614-41cc-bffe-79ed7194bfab-kube-api-access-7qbbl\") pod \"console-operator-68b757fb67-d96q5\" (UID: \"f8286691-e614-41cc-bffe-79ed7194bfab\") " pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.640768 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.640731 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppq8r\" (UniqueName: \"kubernetes.io/projected/31228800-77a6-4869-9090-7cbe6fa30887-kube-api-access-ppq8r\") pod \"service-ca-operator-5d46d9ffd4-mn4xv\" (UID: \"31228800-77a6-4869-9090-7cbe6fa30887\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.640961 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.640939 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbr6\" (UniqueName: \"kubernetes.io/projected/397726ac-cf54-475f-aed6-0bc95cf73ad0-kube-api-access-gvbr6\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:40.641062 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.641023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lqhv\" (UniqueName: \"kubernetes.io/projected/3670b58c-91c7-48c2-98c3-d3dc3ccd010e-kube-api-access-9lqhv\") pod \"volume-data-source-validator-5d456fdbcd-xdq24\" (UID: \"3670b58c-91c7-48c2-98c3-d3dc3ccd010e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" Jul 07 15:44:40.641245 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.641228 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqbq\" (UniqueName: \"kubernetes.io/projected/64e727a5-62e8-49d9-8c1b-9ec2638408ac-kube-api-access-zbqbq\") pod \"network-check-source-989f8b8dc-cm8lr\" (UID: \"64e727a5-62e8-49d9-8c1b-9ec2638408ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" Jul 07 15:44:40.642023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.642008 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv2s8\" (UniqueName: \"kubernetes.io/projected/f702084c-90f6-49db-a2fd-5a5bee9e76be-kube-api-access-gv2s8\") pod \"insights-operator-5c847dcfc4-bvhfr\" (UID: \"f702084c-90f6-49db-a2fd-5a5bee9e76be\") " pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.674298 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.674271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:40.727422 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.727365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-stats-auth\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.727597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.727525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-default-certificate\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.727597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.727553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.727597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.727584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.727748 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.727631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqsjn\" (UniqueName: \"kubernetes.io/projected/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-kube-api-access-zqsjn\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.727803 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.727783 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:41.227758517 +0000 UTC m=+54.458824788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:40.727943 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.727920 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:40.728022 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:40.727989 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:41.227973047 +0000 UTC m=+54.459039312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:40.730033 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.730005 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-stats-auth\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.730139 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.730054 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-default-certificate\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.739077 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.739053 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqsjn\" (UniqueName: \"kubernetes.io/projected/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-kube-api-access-zqsjn\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:40.742864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.742840 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" Jul 07 15:44:40.766425 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.766394 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" Jul 07 15:44:40.767855 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.767829 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf"] Jul 07 15:44:40.781222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.781197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:40.788943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.788917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" Jul 07 15:44:40.810681 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.810653 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" Jul 07 15:44:40.822924 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.822891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:40.935808 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.935765 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv"] Jul 07 15:44:40.968512 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.968422 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bvhfr"] Jul 07 15:44:40.971547 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:40.971522 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf702084c_90f6_49db_a2fd_5a5bee9e76be.slice/crio-470cd08529f54c2bcf658360c75554cbb567ffd635060be8a1076ddee55160b1 WatchSource:0}: Error finding container 470cd08529f54c2bcf658360c75554cbb567ffd635060be8a1076ddee55160b1: Status 404 returned error can't find the container with id 470cd08529f54c2bcf658360c75554cbb567ffd635060be8a1076ddee55160b1 Jul 07 15:44:40.994519 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:40.994484 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24"] Jul 07 15:44:40.997995 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:40.997971 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3670b58c_91c7_48c2_98c3_d3dc3ccd010e.slice/crio-623fe8650459cc6f15302418a13c8ab176ed2afad2b3129ec0682706be25df26 WatchSource:0}: Error finding container 623fe8650459cc6f15302418a13c8ab176ed2afad2b3129ec0682706be25df26: Status 404 returned error can't find the container with id 623fe8650459cc6f15302418a13c8ab176ed2afad2b3129ec0682706be25df26 Jul 07 15:44:41.025065 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.025036 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr"] Jul 07 15:44:41.026970 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:41.026941 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e727a5_62e8_49d9_8c1b_9ec2638408ac.slice/crio-a6ff0af235955643209930b4cbe4de5506eb26f9f942d6e863e759e28ce0b1b8 WatchSource:0}: Error finding container a6ff0af235955643209930b4cbe4de5506eb26f9f942d6e863e759e28ce0b1b8: Status 404 returned error can't find the container with id a6ff0af235955643209930b4cbe4de5506eb26f9f942d6e863e759e28ce0b1b8 Jul 07 15:44:41.032534 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.032511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:41.032665 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.032649 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:41.032717 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.032708 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:42.032692645 +0000 UTC m=+55.263758898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:41.044344 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.044284 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d96q5"] Jul 07 15:44:41.047129 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:41.047105 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8286691_e614_41cc_bffe_79ed7194bfab.slice/crio-5445eaf0a4d6be1433d073a0b914be36e534c8e7af2d73ffb7cf1cd2bc91eb9e WatchSource:0}: Error finding container 5445eaf0a4d6be1433d073a0b914be36e534c8e7af2d73ffb7cf1cd2bc91eb9e: Status 404 returned error can't find the container with id 5445eaf0a4d6be1433d073a0b914be36e534c8e7af2d73ffb7cf1cd2bc91eb9e Jul 07 15:44:41.133946 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.133901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:41.134123 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.134074 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:44:41.134180 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.134155 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls podName:397726ac-cf54-475f-aed6-0bc95cf73ad0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:42.134132899 +0000 UTC m=+55.365199168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8hmgm" (UID: "397726ac-cf54-475f-aed6-0bc95cf73ad0") : secret "samples-operator-tls" not found Jul 07 15:44:41.139780 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.139751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" event={"ID":"f702084c-90f6-49db-a2fd-5a5bee9e76be","Type":"ContainerStarted","Data":"470cd08529f54c2bcf658360c75554cbb567ffd635060be8a1076ddee55160b1"} Jul 07 15:44:41.140750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.140729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" event={"ID":"64e727a5-62e8-49d9-8c1b-9ec2638408ac","Type":"ContainerStarted","Data":"a6ff0af235955643209930b4cbe4de5506eb26f9f942d6e863e759e28ce0b1b8"} Jul 07 15:44:41.141649 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.141623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" event={"ID":"f8286691-e614-41cc-bffe-79ed7194bfab","Type":"ContainerStarted","Data":"5445eaf0a4d6be1433d073a0b914be36e534c8e7af2d73ffb7cf1cd2bc91eb9e"} Jul 07 15:44:41.142758 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.142737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" event={"ID":"3670b58c-91c7-48c2-98c3-d3dc3ccd010e","Type":"ContainerStarted","Data":"623fe8650459cc6f15302418a13c8ab176ed2afad2b3129ec0682706be25df26"} Jul 07 15:44:41.143740 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.143721 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" event={"ID":"31228800-77a6-4869-9090-7cbe6fa30887","Type":"ContainerStarted","Data":"4b840831f58d9c66c258a66e70e4086edd42eb2645e965162d9813117f8b43de"} Jul 07 15:44:41.144648 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.144627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" event={"ID":"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648","Type":"ContainerStarted","Data":"3a4a0154f762f8d2a56fc08e3c44e7baafc801187d52ca5e390101cabf46487f"} Jul 07 15:44:41.234694 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.234663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:41.234811 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:41.234720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:41.234879 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.234831 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:42.234814251 +0000 UTC m=+55.465880503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:41.234927 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.234893 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:41.234962 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:41.234942 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:42.234930837 +0000 UTC m=+55.465997095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:42.043170 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.042584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:42.043170 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.042730 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:42.043170 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.042797 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:44.042776887 +0000 UTC m=+57.273843143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:42.143168 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.143061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:42.143354 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.143234 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:44:42.143354 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.143312 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls podName:397726ac-cf54-475f-aed6-0bc95cf73ad0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:44.14329014 +0000 UTC m=+57.374356399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8hmgm" (UID: "397726ac-cf54-475f-aed6-0bc95cf73ad0") : secret "samples-operator-tls" not found Jul 07 15:44:42.245168 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.244426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:42.245168 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.244495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:42.245168 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.244725 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:42.245168 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.244791 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:44.244771371 +0000 UTC m=+57.475837647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:42.245168 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:42.245073 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:44.245055264 +0000 UTC m=+57.476121528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:42.358111 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.357967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:42.422797 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:42.422765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:44.061371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:44.061331 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:44.061795 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.061499 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:44.061795 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.061580 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.061564975 +0000 UTC m=+61.292631229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:44.162952 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:44.162906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:44.163099 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.163074 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:44:44.163148 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.163138 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls podName:397726ac-cf54-475f-aed6-0bc95cf73ad0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.163122764 +0000 UTC m=+61.394189021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8hmgm" (UID: "397726ac-cf54-475f-aed6-0bc95cf73ad0") : secret "samples-operator-tls" not found Jul 07 15:44:44.263568 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:44.263529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:44.263743 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.263717 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:44.263806 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:44.263728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:44.263806 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.263796 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.263773941 +0000 UTC m=+61.494840211 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:44.263911 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:44.263833 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.263816744 +0000 UTC m=+61.494883000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:45.460906 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:45.460828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:45.997903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:45.997831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:46.101959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:46.101928 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:46.207120 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:46.207093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:47.164771 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.164729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" event={"ID":"f702084c-90f6-49db-a2fd-5a5bee9e76be","Type":"ContainerStarted","Data":"def465cea29318e69212a4ebca47261d8e65976c2113c83ccf51b113c9b682fb"} Jul 07 15:44:47.166399 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.166352 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" event={"ID":"64e727a5-62e8-49d9-8c1b-9ec2638408ac","Type":"ContainerStarted","Data":"19b23d51ede9fff2517fbd0adf04360fab5a23bf3ab287e761632c26e8078c92"} Jul 07 15:44:47.168165 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.168142 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/0.log" Jul 07 15:44:47.168286 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.168185 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8286691-e614-41cc-bffe-79ed7194bfab" containerID="90c9acec250bb5573a02c6bf5ac789a7fb3368b2d3f1b68ffc4cfa31ef8abd05" exitCode=255 Jul 07 15:44:47.168286 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.168251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" event={"ID":"f8286691-e614-41cc-bffe-79ed7194bfab","Type":"ContainerDied","Data":"90c9acec250bb5573a02c6bf5ac789a7fb3368b2d3f1b68ffc4cfa31ef8abd05"} Jul 07 15:44:47.168469 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.168447 2562 scope.go:117] "RemoveContainer" containerID="90c9acec250bb5573a02c6bf5ac789a7fb3368b2d3f1b68ffc4cfa31ef8abd05" Jul 07 15:44:47.170615 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.170588 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" event={"ID":"3670b58c-91c7-48c2-98c3-d3dc3ccd010e","Type":"ContainerStarted","Data":"4d45c2fe3a3992f8758401dffd70c1073ad167647924bc750f6a1051b7f8c77d"} Jul 07 15:44:47.172529 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.172504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" event={"ID":"31228800-77a6-4869-9090-7cbe6fa30887","Type":"ContainerStarted","Data":"ec2e32046931fbc0c68e5809efcd705841a0a2bdc88ea90ff0193e918a0140c1"} Jul 07 15:44:47.174248 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.174223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" event={"ID":"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648","Type":"ContainerStarted","Data":"09691c5da7a55b5a653139734507453eda6a13877d0bbe4e5f54847df83ee10f"} Jul 07 15:44:47.282033 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.281981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:47.286666 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.286588 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" podStartSLOduration=2.09716703 podStartE2EDuration="7.286565705s" totalImagesPullingTime="5.189398675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:47.28152197 +0000 UTC m=+60.512588246" watchObservedRunningTime="2026-07-07 15:44:47.286565705 +0000 UTC m=+60.517631980" Jul 07 15:44:47.368487 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.368327 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xdq24" podStartSLOduration=2.207379291 podStartE2EDuration="7.368306513s" totalImagesPullingTime="5.160927222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:47.367118803 +0000 UTC m=+60.598185080" watchObservedRunningTime="2026-07-07 15:44:47.368306513 +0000 UTC m=+60.599372789" Jul 07 15:44:47.368793 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.368751 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" podStartSLOduration=2.157467835 podStartE2EDuration="7.368739028s" totalImagesPullingTime="5.211271193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:47.334561717 +0000 UTC m=+60.565627991" watchObservedRunningTime="2026-07-07 15:44:47.368739028 +0000 UTC m=+60.599805305" Jul 07 15:44:47.397454 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.397416 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8b6q9"] Jul 07 15:44:47.401027 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.401005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.408042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.406500 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:44:47.408042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.406500 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:44:47.408042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.406571 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsn42\"" Jul 07 15:44:47.413393 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.413285 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-cm8lr" podStartSLOduration=2.262076899 podStartE2EDuration="7.41326407s" totalImagesPullingTime="5.151187171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:47.413051141 +0000 UTC m=+60.644117417" watchObservedRunningTime="2026-07-07 15:44:47.41326407 +0000 UTC m=+60.644330346" Jul 07 15:44:47.416430 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.416346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:47.421635 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.421613 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8b6q9"] Jul 07 15:44:47.480578 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.480514 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" podStartSLOduration=2.083823363 podStartE2EDuration="7.480495238s" totalImagesPullingTime="5.396671875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:47.479764078 +0000 UTC m=+60.710830353" watchObservedRunningTime="2026-07-07 15:44:47.480495238 +0000 UTC m=+60.711561514" Jul 07 15:44:47.496492 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.496454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.496785 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.496764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ld5\" (UniqueName: \"kubernetes.io/projected/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-api-access-p9ld5\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.497056 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.497034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/034ee060-6915-4f0e-a3e5-904d28a71df2-crio-socket\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.497210 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.497194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/034ee060-6915-4f0e-a3e5-904d28a71df2-data-volume\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.497392 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.497362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.520929 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.520892 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:47.592974 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.592941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:47.598277 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/034ee060-6915-4f0e-a3e5-904d28a71df2-crio-socket\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/034ee060-6915-4f0e-a3e5-904d28a71df2-data-volume\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598330 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598400 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/034ee060-6915-4f0e-a3e5-904d28a71df2-crio-socket\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598648 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ld5\" (UniqueName: \"kubernetes.io/projected/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-api-access-p9ld5\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.598648 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:47.598471 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:44:47.598648 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:47.598635 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.098612031 +0000 UTC m=+61.329678299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : configmap "kube-rbac-proxy" not found Jul 07 15:44:47.598648 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:47.598552 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:44:47.598814 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:47.598705 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:48.098688411 +0000 UTC m=+61.329754678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : secret "insights-runtime-extractor-tls" not found Jul 07 15:44:47.598814 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.598706 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/034ee060-6915-4f0e-a3e5-904d28a71df2-data-volume\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.608580 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.608553 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ld5\" (UniqueName: \"kubernetes.io/projected/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-api-access-p9ld5\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:47.662365 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:47.662326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.093690 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.093650 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.103766 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.103743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:48.103870 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.103799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:48.103908 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.103883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:48.103984 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.103886 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:48.104026 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.103939 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:44:48.104069 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.104028 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:56.104010197 +0000 UTC m=+69.335076451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:48.104069 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.103942 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:44:48.104069 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.104045 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:49.104035979 +0000 UTC m=+62.335102232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : secret "insights-runtime-extractor-tls" not found Jul 07 15:44:48.104069 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.104064 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:49.10405297 +0000 UTC m=+62.335119238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : configmap "kube-rbac-proxy" not found Jul 07 15:44:48.179569 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.179540 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/1.log" Jul 07 15:44:48.179966 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.179943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/0.log" Jul 07 15:44:48.180011 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.179976 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8286691-e614-41cc-bffe-79ed7194bfab" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" exitCode=255 Jul 07 15:44:48.180054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.180020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" event={"ID":"f8286691-e614-41cc-bffe-79ed7194bfab","Type":"ContainerDied","Data":"44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1"} Jul 07 15:44:48.180096 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.180074 2562 scope.go:117] "RemoveContainer" containerID="90c9acec250bb5573a02c6bf5ac789a7fb3368b2d3f1b68ffc4cfa31ef8abd05" Jul 07 15:44:48.180319 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.180270 2562 scope.go:117] "RemoveContainer" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" Jul 07 15:44:48.180484 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.180465 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:44:48.193940 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.193915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.205195 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.205165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:48.205322 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.205302 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:44:48.205401 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.205364 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls podName:397726ac-cf54-475f-aed6-0bc95cf73ad0 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:56.205349263 +0000 UTC m=+69.436415516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8hmgm" (UID: "397726ac-cf54-475f-aed6-0bc95cf73ad0") : secret "samples-operator-tls" not found Jul 07 15:44:48.307081 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.307039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:48.307808 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.307776 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:56.307752709 +0000 UTC m=+69.538818962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:48.308029 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.307889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:48.308139 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.308105 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:48.308236 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:48.308156 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:44:56.308139592 +0000 UTC m=+69.539205851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:48.352706 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.352627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.452440 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.452405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.570471 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.570438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:48.669864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:48.669785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.081914 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.081889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4qpz" Jul 07 15:44:49.116516 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.116480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:49.116681 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:49.116661 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:44:49.116765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.116680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:49.116765 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:49.116731 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:51.116712845 +0000 UTC m=+64.347779115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : secret "insights-runtime-extractor-tls" not found Jul 07 15:44:49.116765 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:49.116752 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:44:49.116927 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:49.116815 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:51.116801813 +0000 UTC m=+64.347868078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : configmap "kube-rbac-proxy" not found Jul 07 15:44:49.184627 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.184597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/1.log" Jul 07 15:44:49.185049 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.185025 2562 scope.go:117] "RemoveContainer" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" Jul 07 15:44:49.185277 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:49.185245 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:44:49.257701 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.257656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.347971 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.347881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.428532 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.428500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.802392 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.802349 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sphrk_6be354b0-ab22-49c7-826b-60a7886c2f21/dns-node-resolver/0.log" Jul 07 15:44:49.832856 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.832825 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.933024 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.932990 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:49.952518 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.952483 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gzhk6"] Jul 07 15:44:49.987494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.987459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gzhk6"] Jul 07 15:44:49.987641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.987581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:49.990433 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.990410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wxzvc\"" Jul 07 15:44:49.990674 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.990659 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 15:44:49.990777 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.990740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 15:44:49.991576 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.991560 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 15:44:49.993752 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:49.993662 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 15:44:50.015790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.015756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:50.023892 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.023868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-cabundle\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.024033 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.023898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-key\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.024103 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.024026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz7tt\" (UniqueName: \"kubernetes.io/projected/94472357-2728-4c44-b6a7-b2c478ff6f96-kube-api-access-sz7tt\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.125453 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.125329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz7tt\" (UniqueName: \"kubernetes.io/projected/94472357-2728-4c44-b6a7-b2c478ff6f96-kube-api-access-sz7tt\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.125622 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.125521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-cabundle\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.125622 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.125557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-key\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.126155 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.126132 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-cabundle\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.127946 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.127926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94472357-2728-4c44-b6a7-b2c478ff6f96-signing-key\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.134534 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.134513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz7tt\" (UniqueName: \"kubernetes.io/projected/94472357-2728-4c44-b6a7-b2c478ff6f96-kube-api-access-sz7tt\") pod \"service-ca-758f599c5-gzhk6\" (UID: \"94472357-2728-4c44-b6a7-b2c478ff6f96\") " pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.296023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.295989 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-gzhk6" Jul 07 15:44:50.442928 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.442887 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gzhk6"] Jul 07 15:44:50.446450 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:50.446425 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94472357_2728_4c44_b6a7_b2c478ff6f96.slice/crio-43c0efd08b7869ae5829b63e8e0daa1581198b974b0145ccb445cea6d8ffa1ec WatchSource:0}: Error finding container 43c0efd08b7869ae5829b63e8e0daa1581198b974b0145ccb445cea6d8ffa1ec: Status 404 returned error can't find the container with id 43c0efd08b7869ae5829b63e8e0daa1581198b974b0145ccb445cea6d8ffa1ec Jul 07 15:44:50.823534 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.823500 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:50.823731 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.823557 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:44:50.823870 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.823857 2562 scope.go:117] "RemoveContainer" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" Jul 07 15:44:50.824021 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:50.824004 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:44:50.973021 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.972979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:50.994062 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:50.994038 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8q4v_9b02124c-6de0-4c0e-83ed-914d2142429c/node-ca/0.log" Jul 07 15:44:51.135233 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.135150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:51.135386 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:51.135292 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:44:51.135386 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.135346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:51.135386 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:51.135355 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:55.135338622 +0000 UTC m=+68.366404875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : secret "insights-runtime-extractor-tls" not found Jul 07 15:44:51.135493 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:51.135448 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:44:51.135530 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:51.135501 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:44:55.135488607 +0000 UTC m=+68.366554860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : configmap "kube-rbac-proxy" not found Jul 07 15:44:51.193060 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.193026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-gzhk6" event={"ID":"94472357-2728-4c44-b6a7-b2c478ff6f96","Type":"ContainerStarted","Data":"2a71d65d0f2b51389e91f0647be5bb9b57d8ceb70cdf8a56ba07963378c41fc1"} Jul 07 15:44:51.193244 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.193071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-gzhk6" event={"ID":"94472357-2728-4c44-b6a7-b2c478ff6f96","Type":"ContainerStarted","Data":"43c0efd08b7869ae5829b63e8e0daa1581198b974b0145ccb445cea6d8ffa1ec"} Jul 07 15:44:51.210838 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.210782 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-gzhk6" podStartSLOduration=2.210765813 podStartE2EDuration="2.210765813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:51.208963397 +0000 UTC m=+64.440029695" watchObservedRunningTime="2026-07-07 15:44:51.210765813 +0000 UTC m=+64.441832088" Jul 07 15:44:51.391473 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:51.391395 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5rmh7_24bbd909-d7bd-453e-83eb-6c1d5120b132/serve-healthcheck-canary/0.log" Jul 07 15:44:52.164146 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.164112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:52.192098 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.192058 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q8xlf_cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648/kube-storage-version-migrator-operator/0.log" Jul 07 15:44:52.288860 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.288819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:52.389488 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.389454 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:52.490162 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.490065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:52.599800 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.599757 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:52.696849 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:52.696795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:53.070861 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.070828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:53.153494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.153455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:44:53.156735 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.156720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:44:53.163968 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:53.163953 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 15:44:53.164038 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:53.164009 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs podName:639ebdf2-6bbe-42a5-8ade-322a278cad2d nodeName:}" failed. No retries permitted until 2026-07-07 15:45:57.163993174 +0000 UTC m=+130.395059426 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs") pod "network-metrics-daemon-42jm9" (UID: "639ebdf2-6bbe-42a5-8ade-322a278cad2d") : secret "metrics-daemon-secret" not found Jul 07 15:44:53.254843 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.254803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:53.257887 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.257869 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:44:53.267592 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.267565 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de821d61-5eb5-47bd-a9ce-24fe4b4871f6-original-pull-secret\") pod \"global-pull-secret-syncer-9mlh8\" (UID: \"de821d61-5eb5-47bd-a9ce-24fe4b4871f6\") " pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:53.355869 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.355780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:53.358359 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.358325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sszkf\" (UniqueName: \"kubernetes.io/projected/e10b46df-00c3-4213-97ca-84bd168cd7b3-kube-api-access-sszkf\") pod \"network-check-target-qbdb7\" (UID: \"e10b46df-00c3-4213-97ca-84bd168cd7b3\") " pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:53.463933 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.463883 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:53.532190 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.532140 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9mlh8" Jul 07 15:44:53.543623 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.543597 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ctzps\"" Jul 07 15:44:53.551195 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.551172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:53.674751 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.674702 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9mlh8"] Jul 07 15:44:53.677242 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:53.677211 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde821d61_5eb5_47bd_a9ce_24fe4b4871f6.slice/crio-937d2b99f29dc455a777acd4c79fdb527306ef0c9e00c53e61a9fbc57fffbfb9 WatchSource:0}: Error finding container 937d2b99f29dc455a777acd4c79fdb527306ef0c9e00c53e61a9fbc57fffbfb9: Status 404 returned error can't find the container with id 937d2b99f29dc455a777acd4c79fdb527306ef0c9e00c53e61a9fbc57fffbfb9 Jul 07 15:44:53.689935 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:53.689911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbdb7"] Jul 07 15:44:53.693610 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:44:53.693584 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10b46df_00c3_4213_97ca_84bd168cd7b3.slice/crio-a2565b4ac95bc2c0115be273bd3333eb827b5ab89a09126be1eb0b3606b96abd WatchSource:0}: Error finding container a2565b4ac95bc2c0115be273bd3333eb827b5ab89a09126be1eb0b3606b96abd: Status 404 returned error can't find the container with id a2565b4ac95bc2c0115be273bd3333eb827b5ab89a09126be1eb0b3606b96abd Jul 07 15:44:54.078685 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.078656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:54.209750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.209705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9mlh8" event={"ID":"de821d61-5eb5-47bd-a9ce-24fe4b4871f6","Type":"ContainerStarted","Data":"937d2b99f29dc455a777acd4c79fdb527306ef0c9e00c53e61a9fbc57fffbfb9"} Jul 07 15:44:54.211203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.211177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbdb7" event={"ID":"e10b46df-00c3-4213-97ca-84bd168cd7b3","Type":"ContainerStarted","Data":"bd97a4523b78ae8bdc41f3ceb3af40973ef0f973f1b4eac18d88c18dd6d1c8d1"} Jul 07 15:44:54.211340 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.211217 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbdb7" event={"ID":"e10b46df-00c3-4213-97ca-84bd168cd7b3","Type":"ContainerStarted","Data":"a2565b4ac95bc2c0115be273bd3333eb827b5ab89a09126be1eb0b3606b96abd"} Jul 07 15:44:54.211340 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.211307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:44:54.230555 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.230506 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qbdb7" podStartSLOduration=66.230491267 podStartE2EDuration="1m6.230491267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:54.23025573 +0000 UTC m=+67.461322005" watchObservedRunningTime="2026-07-07 15:44:54.230491267 +0000 UTC m=+67.461557535" Jul 07 15:44:54.568252 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.568214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:44:54.568252 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.568263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:44:54.568763 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:54.568404 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:44:54.568763 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:54.568474 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls podName:401d0f3f-db12-451d-8452-d3348ccd8de9 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:26.568454209 +0000 UTC m=+99.799520483 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls") pod "dns-default-2kbdn" (UID: "401d0f3f-db12-451d-8452-d3348ccd8de9") : secret "dns-default-metrics-tls" not found Jul 07 15:44:54.568763 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:54.568485 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:44:54.568763 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:54.568508 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b666b688f-8cm8m: secret "image-registry-tls" not found Jul 07 15:44:54.568763 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:54.568564 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls podName:e463e2ad-2e40-4669-b119-f0f6e69f6f8b nodeName:}" failed. No retries permitted until 2026-07-07 15:45:26.568543315 +0000 UTC m=+99.799609570 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls") pod "image-registry-6b666b688f-8cm8m" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b") : secret "image-registry-tls" not found Jul 07 15:44:54.672149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.672110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:54.875302 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:54.875214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:55.174277 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.174193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:55.174464 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.174291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:44:55.174464 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:55.174331 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:44:55.174464 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:55.174421 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:03.174399966 +0000 UTC m=+76.405466223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : configmap "kube-rbac-proxy" not found Jul 07 15:44:55.174605 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:55.174479 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:44:55.174605 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:55.174535 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls podName:034ee060-6915-4f0e-a3e5-904d28a71df2 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:03.174519422 +0000 UTC m=+76.405585681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8b6q9" (UID: "034ee060-6915-4f0e-a3e5-904d28a71df2") : secret "insights-runtime-extractor-tls" not found Jul 07 15:44:55.268962 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.268921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:55.487972 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.487888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:55.568761 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.568722 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:55.676093 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.676059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:55.874449 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:55.874415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:56.136671 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.136587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:56.185730 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.185687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:44:56.185899 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:56.185862 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:56.185950 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:56.185941 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls podName:b2b5f3bd-1764-4092-a798-cf68777254c4 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:12.185920636 +0000 UTC m=+85.416986892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q75ps" (UID: "b2b5f3bd-1764-4092-a798-cf68777254c4") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:44:56.286563 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.286525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:56.289263 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.289216 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/397726ac-cf54-475f-aed6-0bc95cf73ad0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8hmgm\" (UID: \"397726ac-cf54-475f-aed6-0bc95cf73ad0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:56.353029 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.352999 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-z74qr\"" Jul 07 15:44:56.360594 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.360565 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" Jul 07 15:44:56.387567 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.387498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:56.387567 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.387551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:44:56.387745 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:56.387675 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:45:12.387656763 +0000 UTC m=+85.618723039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : configmap references non-existent config key: service-ca.crt Jul 07 15:44:56.387745 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:56.387687 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:44:56.387745 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:44:56.387739 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs podName:ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f nodeName:}" failed. No retries permitted until 2026-07-07 15:45:12.387722025 +0000 UTC m=+85.618788301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs") pod "router-default-d6c7b58d5-x9h75" (UID: "ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f") : secret "router-metrics-certs-default" not found Jul 07 15:44:56.479255 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.479174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:56.515876 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.515839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm"] Jul 07 15:44:56.560536 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.560505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:56.857765 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:56.857729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:57.226706 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:57.226623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" event={"ID":"397726ac-cf54-475f-aed6-0bc95cf73ad0","Type":"ContainerStarted","Data":"be724868cf2b11ccc2a76a682d5be41ca3e8a32b64acb5e1c8294c11d2635a5c"} Jul 07 15:44:57.266728 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:57.266692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:57.661755 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:57.661691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:57.998881 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:57.998770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:58.102541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:58.102496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:58.267608 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:58.267542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:58.664163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:58.664123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:59.235251 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:59.235215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9mlh8" event={"ID":"de821d61-5eb5-47bd-a9ce-24fe4b4871f6","Type":"ContainerStarted","Data":"69216d77bfd16f5ab9ddb7bb67023c5af9db84b948201ad59e440f5b7d7f2661"} Jul 07 15:44:59.251904 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:59.251854 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9mlh8" podStartSLOduration=65.612118453 podStartE2EDuration="1m10.251838953s" totalImagesPullingTime="4.6397205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:59.250695835 +0000 UTC m=+72.481762317" watchObservedRunningTime="2026-07-07 15:44:59.251838953 +0000 UTC m=+72.482905229" Jul 07 15:44:59.256177 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:59.256151 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:44:59.790078 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:44:59.790043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:00.079233 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.079122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:00.241881 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.241842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" event={"ID":"397726ac-cf54-475f-aed6-0bc95cf73ad0","Type":"ContainerStarted","Data":"71218e9f3aa3b42c82a1d5f96f18753d49f30bd2dd51063d92733c5d7d8e3aa4"} Jul 07 15:45:00.242336 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.241910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" event={"ID":"397726ac-cf54-475f-aed6-0bc95cf73ad0","Type":"ContainerStarted","Data":"5e4304b7c111edda8181bc58842a2fa6ee5f3159dc2b7a8be21ee261b8b0d80a"} Jul 07 15:45:00.261505 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.261449 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8hmgm" podStartSLOduration=17.026900384 podStartE2EDuration="20.261433553s" totalImagesPullingTime="3.234533169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:00.260933503 +0000 UTC m=+73.491999799" watchObservedRunningTime="2026-07-07 15:45:00.261433553 +0000 UTC m=+73.492499828" Jul 07 15:45:00.280474 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.280446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:00.389584 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.389458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:00.668676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:00.668569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:01.456370 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:01.456338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:01.732027 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:01.731915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:01.833876 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:01.833840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:02.058307 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:02.058247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:02.263721 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:02.263690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:02.663079 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:02.663043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:02.890429 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:02.890362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:02.920987 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:02.920921 2562 scope.go:117] "RemoveContainer" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" Jul 07 15:45:03.251317 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.251225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:45:03.251317 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.251318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:45:03.251870 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.251849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/034ee060-6915-4f0e-a3e5-904d28a71df2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:45:03.253457 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.253440 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:45:03.253848 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.253822 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/034ee060-6915-4f0e-a3e5-904d28a71df2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8b6q9\" (UID: \"034ee060-6915-4f0e-a3e5-904d28a71df2\") " pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:45:03.253934 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.253878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/1.log" Jul 07 15:45:03.253934 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.253910 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8286691-e614-41cc-bffe-79ed7194bfab" containerID="de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529" exitCode=255 Jul 07 15:45:03.254006 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.253980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" event={"ID":"f8286691-e614-41cc-bffe-79ed7194bfab","Type":"ContainerDied","Data":"de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529"} Jul 07 15:45:03.254047 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.254037 2562 scope.go:117] "RemoveContainer" containerID="44ecd01d44802bd7fd5c7f37783a560d007ac1904e7e2ceb2d8fa92a462d6aa1" Jul 07 15:45:03.254435 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.254414 2562 scope.go:117] "RemoveContainer" containerID="de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529" Jul 07 15:45:03.254594 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:03.254570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:45:03.316513 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.316483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8b6q9" Jul 07 15:45:03.444004 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:03.443973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8b6q9"] Jul 07 15:45:03.447288 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:03.447256 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034ee060_6915_4f0e_a3e5_904d28a71df2.slice/crio-c5b03f83aa279781079675d8da18707f18ac2520c1980b64cd3071ba922f7209 WatchSource:0}: Error finding container c5b03f83aa279781079675d8da18707f18ac2520c1980b64cd3071ba922f7209: Status 404 returned error can't find the container with id c5b03f83aa279781079675d8da18707f18ac2520c1980b64cd3071ba922f7209 Jul 07 15:45:04.260834 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:04.260806 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:45:04.262494 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:04.262463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8b6q9" event={"ID":"034ee060-6915-4f0e-a3e5-904d28a71df2","Type":"ContainerStarted","Data":"c5b03f83aa279781079675d8da18707f18ac2520c1980b64cd3071ba922f7209"} Jul 07 15:45:08.280724 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:08.280689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8b6q9" event={"ID":"034ee060-6915-4f0e-a3e5-904d28a71df2","Type":"ContainerStarted","Data":"694c024042aee4ce924ce6c90552cb3a49340d2f03aa688a05540024a90b4e51"} Jul 07 15:45:08.281095 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:08.280747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8b6q9" event={"ID":"034ee060-6915-4f0e-a3e5-904d28a71df2","Type":"ContainerStarted","Data":"332822c2e6de00f922c31c0c1f8ccd4158632312f9c7e86c4b655f9c30adb2bb"} Jul 07 15:45:10.823760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:10.823665 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:45:10.823760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:10.823730 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:45:10.824165 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:10.824046 2562 scope.go:117] "RemoveContainer" containerID="de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529" Jul 07 15:45:10.824222 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:10.824200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:45:11.293429 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:11.293373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8b6q9" event={"ID":"034ee060-6915-4f0e-a3e5-904d28a71df2","Type":"ContainerStarted","Data":"eb74bf10bf12785d123c64bab8768f854a7d87fb98d9e4d5681e993dd89a407b"} Jul 07 15:45:11.315037 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:11.314986 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8b6q9" podStartSLOduration=17.316457275 podStartE2EDuration="24.314969901s" totalImagesPullingTime="6.998512626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:44:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:11.313629647 +0000 UTC m=+84.544695932" watchObservedRunningTime="2026-07-07 15:45:11.314969901 +0000 UTC m=+84.546036175" Jul 07 15:45:12.232393 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.232337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:45:12.234771 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.234750 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2b5f3bd-1764-4092-a798-cf68777254c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q75ps\" (UID: \"b2b5f3bd-1764-4092-a798-cf68777254c4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:45:12.433313 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.433266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:12.433579 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.433555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:12.434148 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.434131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-service-ca-bundle\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:12.435649 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.435632 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f-metrics-certs\") pod \"router-default-d6c7b58d5-x9h75\" (UID: \"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f\") " pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:12.519082 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.519000 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xmr7f\"" Jul 07 15:45:12.526635 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.526611 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" Jul 07 15:45:12.647695 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.647666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tqbd5\"" Jul 07 15:45:12.652802 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.652775 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps"] Jul 07 15:45:12.655372 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.655341 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:12.655725 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:12.655697 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b5f3bd_1764_4092_a798_cf68777254c4.slice/crio-c1d9eb7ea1328d85402a0135ab8831757ced98f1418e7326c2097b768dcdea4f WatchSource:0}: Error finding container c1d9eb7ea1328d85402a0135ab8831757ced98f1418e7326c2097b768dcdea4f: Status 404 returned error can't find the container with id c1d9eb7ea1328d85402a0135ab8831757ced98f1418e7326c2097b768dcdea4f Jul 07 15:45:12.780821 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.780742 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:12.784824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.784797 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d6c7b58d5-x9h75"] Jul 07 15:45:12.787765 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:12.787738 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9fda8e_c1fc_41b9_831d_e5ec43d8e94f.slice/crio-f7b7d6a8d5b19309021eca882abcd9a148aaeeb4680124ee96e1ed5c79757b2a WatchSource:0}: Error finding container f7b7d6a8d5b19309021eca882abcd9a148aaeeb4680124ee96e1ed5c79757b2a: Status 404 returned error can't find the container with id f7b7d6a8d5b19309021eca882abcd9a148aaeeb4680124ee96e1ed5c79757b2a Jul 07 15:45:12.884193 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:12.884161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:13.300861 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.300824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" event={"ID":"b2b5f3bd-1764-4092-a798-cf68777254c4","Type":"ContainerStarted","Data":"c1d9eb7ea1328d85402a0135ab8831757ced98f1418e7326c2097b768dcdea4f"} Jul 07 15:45:13.302320 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.302296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" event={"ID":"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f","Type":"ContainerStarted","Data":"8158c039df7b30c0e4b7847fe25f0edf3f55671b7ae84e488631c73c20da26b8"} Jul 07 15:45:13.302456 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.302333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" event={"ID":"ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f","Type":"ContainerStarted","Data":"f7b7d6a8d5b19309021eca882abcd9a148aaeeb4680124ee96e1ed5c79757b2a"} Jul 07 15:45:13.463395 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.463341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:13.576867 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.576777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:13.656046 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.656004 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:13.658856 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.658825 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:13.694016 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.693979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:13.703319 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.703272 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" podStartSLOduration=33.703255893 podStartE2EDuration="33.703255893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:13.346593457 +0000 UTC m=+86.577659760" watchObservedRunningTime="2026-07-07 15:45:13.703255893 +0000 UTC m=+86.934322172" Jul 07 15:45:13.861888 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.861798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:13.979485 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:13.979450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:14.306320 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:14.306287 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:14.307766 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:14.307743 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-d6c7b58d5-x9h75" Jul 07 15:45:14.967658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:14.967632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:15.068996 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.068969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:15.311602 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.311564 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" event={"ID":"b2b5f3bd-1764-4092-a798-cf68777254c4","Type":"ContainerStarted","Data":"12e32aa51d411dc63549a1f88aad749a8aa20e67110f597e956318a6aac99553"} Jul 07 15:45:15.354928 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.354871 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q75ps" podStartSLOduration=33.05239836 podStartE2EDuration="35.354853311s" totalImagesPullingTime="2.302454951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:15.354491314 +0000 UTC m=+88.585557589" watchObservedRunningTime="2026-07-07 15:45:15.354853311 +0000 UTC m=+88.585919586" Jul 07 15:45:15.480752 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.480716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:15.581068 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.580981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:15.630810 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.630773 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76"] Jul 07 15:45:15.648830 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.648800 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76"] Jul 07 15:45:15.648974 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.648900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:15.651993 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.651968 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 15:45:15.652136 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.651990 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vrsrq\"" Jul 07 15:45:15.663038 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.663015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7c76\" (UID: \"3b5380cb-269b-40b1-8868-e1507d5bc2f3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:15.673259 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.673230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:15.763717 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.763678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7c76\" (UID: \"3b5380cb-269b-40b1-8868-e1507d5bc2f3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:15.763903 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:15.763846 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:45:15.763971 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:15.763926 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates podName:3b5380cb-269b-40b1-8868-e1507d5bc2f3 nodeName:}" failed. No retries permitted until 2026-07-07 15:45:16.263904669 +0000 UTC m=+89.494970922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-h7c76" (UID: "3b5380cb-269b-40b1-8868-e1507d5bc2f3") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:45:15.926961 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:15.926875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:16.267884 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:16.267790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7c76\" (UID: \"3b5380cb-269b-40b1-8868-e1507d5bc2f3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:16.270304 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:16.270278 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b5380cb-269b-40b1-8868-e1507d5bc2f3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7c76\" (UID: \"3b5380cb-269b-40b1-8868-e1507d5bc2f3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:16.558487 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:16.558453 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:16.717222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:16.717192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76"] Jul 07 15:45:16.719971 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:16.719945 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5380cb_269b_40b1_8868_e1507d5bc2f3.slice/crio-2f89899b5747d74beb1f9eb0add613642551e91c819783ca5e24ecf6c7dd9b31 WatchSource:0}: Error finding container 2f89899b5747d74beb1f9eb0add613642551e91c819783ca5e24ecf6c7dd9b31: Status 404 returned error can't find the container with id 2f89899b5747d74beb1f9eb0add613642551e91c819783ca5e24ecf6c7dd9b31 Jul 07 15:45:17.319297 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:17.319260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" event={"ID":"3b5380cb-269b-40b1-8868-e1507d5bc2f3","Type":"ContainerStarted","Data":"2f89899b5747d74beb1f9eb0add613642551e91c819783ca5e24ecf6c7dd9b31"} Jul 07 15:45:19.040646 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.040615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:19.115750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.115717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:19.327040 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.326957 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" event={"ID":"3b5380cb-269b-40b1-8868-e1507d5bc2f3","Type":"ContainerStarted","Data":"28a01aa9541f11fdbac7be9055808ddb02f44497c3721195926826c4c7cfc977"} Jul 07 15:45:19.327184 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.327123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:19.331786 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.331759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" Jul 07 15:45:19.346063 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.346017 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7c76" podStartSLOduration=2.663044322 podStartE2EDuration="4.346002212s" totalImagesPullingTime="1.68295789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:19.34522136 +0000 UTC m=+92.576287636" watchObservedRunningTime="2026-07-07 15:45:19.346002212 +0000 UTC m=+92.577068523" Jul 07 15:45:19.683030 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.682948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:19.809804 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.809735 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:19.918967 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:19.918930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:22.510131 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:22.510099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:22.920728 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:22.920695 2562 scope.go:117] "RemoveContainer" containerID="de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529" Jul 07 15:45:22.920926 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:22.920893 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-d96q5_openshift-console-operator(f8286691-e614-41cc-bffe-79ed7194bfab)\"" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podUID="f8286691-e614-41cc-bffe-79ed7194bfab" Jul 07 15:45:23.193374 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:23.193281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:23.293778 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:23.293741 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:24.037530 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:24.037495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:24.118299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:24.118270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:25.217136 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:25.217105 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qbdb7" Jul 07 15:45:25.445230 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:25.445192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:26.452258 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.452223 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:26.552118 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.552078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:26.652459 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.652425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:45:26.652640 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.652469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:45:26.654881 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.654852 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/401d0f3f-db12-451d-8452-d3348ccd8de9-metrics-tls\") pod \"dns-default-2kbdn\" (UID: \"401d0f3f-db12-451d-8452-d3348ccd8de9\") " pod="openshift-dns/dns-default-2kbdn" Jul 07 15:45:26.655046 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.655028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"image-registry-6b666b688f-8cm8m\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:45:26.823629 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.823589 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fglhg\"" Jul 07 15:45:26.830614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.830586 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:45:26.845624 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.845596 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j6trp\"" Jul 07 15:45:26.853225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:26.853194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2kbdn" Jul 07 15:45:27.005155 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.005115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.033728 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:27.033694 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode463e2ad_2e40_4669_b119_f0f6e69f6f8b.slice/crio-b0c35f2f1b02e6ded55197cc31e45b36016cc8e8ebc0203a0d538f949e75a3f9 WatchSource:0}: Error finding container b0c35f2f1b02e6ded55197cc31e45b36016cc8e8ebc0203a0d538f949e75a3f9: Status 404 returned error can't find the container with id b0c35f2f1b02e6ded55197cc31e45b36016cc8e8ebc0203a0d538f949e75a3f9 Jul 07 15:45:27.035050 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.035029 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:45:27.037864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.037843 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2kbdn"] Jul 07 15:45:27.039590 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:27.039567 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401d0f3f_db12_451d_8452_d3348ccd8de9.slice/crio-70b4e6b7d0b533a28a7162b9dcbf149791c25fc56c5441dd34f63273d9a209ea WatchSource:0}: Error finding container 70b4e6b7d0b533a28a7162b9dcbf149791c25fc56c5441dd34f63273d9a209ea: Status 404 returned error can't find the container with id 70b4e6b7d0b533a28a7162b9dcbf149791c25fc56c5441dd34f63273d9a209ea Jul 07 15:45:27.127838 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.127799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.271125 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.271024 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.358328 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.358238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2kbdn" event={"ID":"401d0f3f-db12-451d-8452-d3348ccd8de9","Type":"ContainerStarted","Data":"70b4e6b7d0b533a28a7162b9dcbf149791c25fc56c5441dd34f63273d9a209ea"} Jul 07 15:45:27.360331 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.360281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" event={"ID":"e463e2ad-2e40-4669-b119-f0f6e69f6f8b","Type":"ContainerStarted","Data":"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7"} Jul 07 15:45:27.360505 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.360341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" event={"ID":"e463e2ad-2e40-4669-b119-f0f6e69f6f8b","Type":"ContainerStarted","Data":"b0c35f2f1b02e6ded55197cc31e45b36016cc8e8ebc0203a0d538f949e75a3f9"} Jul 07 15:45:27.360505 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.360484 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:45:27.399645 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.399605 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-srf2g"] Jul 07 15:45:27.403545 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.403520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.413325 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.412928 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:45:27.413325 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.412943 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:45:27.413325 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.412981 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:45:27.413727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.413597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.416286 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.414460 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l97nq\"" Jul 07 15:45:27.416286 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.414543 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" podStartSLOduration=98.414525733 podStartE2EDuration="1m38.414525733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:27.414445463 +0000 UTC m=+100.645511739" watchObservedRunningTime="2026-07-07 15:45:27.414525733 +0000 UTC m=+100.645592010" Jul 07 15:45:27.416286 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.414637 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:45:27.460673 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-sys\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-textfile\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-wtmp\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-metrics-client-ca\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-root\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.461149 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.460964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qct59\" (UniqueName: \"kubernetes.io/projected/6b8ce45a-0231-4117-b498-559ea6fef53a-kube-api-access-qct59\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561709 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-sys\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561709 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-textfile\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561947 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561784 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-sys\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561947 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-wtmp\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561947 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-metrics-client-ca\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561947 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-root\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.561947 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.561976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-wtmp\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562016 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b8ce45a-0231-4117-b498-559ea6fef53a-root\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qct59\" (UniqueName: \"kubernetes.io/projected/6b8ce45a-0231-4117-b498-559ea6fef53a-kube-api-access-qct59\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:27.562074 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562188 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:45:27.562156 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls podName:6b8ce45a-0231-4117-b498-559ea6fef53a nodeName:}" failed. No retries permitted until 2026-07-07 15:45:28.062136804 +0000 UTC m=+101.293203071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls") pod "node-exporter-srf2g" (UID: "6b8ce45a-0231-4117-b498-559ea6fef53a") : secret "node-exporter-tls" not found Jul 07 15:45:27.562560 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562400 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-textfile\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562825 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562801 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.562907 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.562881 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce45a-0231-4117-b498-559ea6fef53a-metrics-client-ca\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.564470 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.564449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.576949 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.576930 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qct59\" (UniqueName: \"kubernetes.io/projected/6b8ce45a-0231-4117-b498-559ea6fef53a-kube-api-access-qct59\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:27.578028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.578004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.722726 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.722645 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.828614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.828575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:27.922965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:27.922899 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:28.067723 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.067689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:28.070453 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.070425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b8ce45a-0231-4117-b498-559ea6fef53a-node-exporter-tls\") pod \"node-exporter-srf2g\" (UID: \"6b8ce45a-0231-4117-b498-559ea6fef53a\") " pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:28.154985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.154951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:28.254269 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.254226 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:28.317524 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.317487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srf2g" Jul 07 15:45:28.329529 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:28.329492 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8ce45a_0231_4117_b498_559ea6fef53a.slice/crio-74700b7ed513d24a8f9958533ccf039725fdc077882dc2d2d5f9f987eec73561 WatchSource:0}: Error finding container 74700b7ed513d24a8f9958533ccf039725fdc077882dc2d2d5f9f987eec73561: Status 404 returned error can't find the container with id 74700b7ed513d24a8f9958533ccf039725fdc077882dc2d2d5f9f987eec73561 Jul 07 15:45:28.365592 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.365550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srf2g" event={"ID":"6b8ce45a-0231-4117-b498-559ea6fef53a","Type":"ContainerStarted","Data":"74700b7ed513d24a8f9958533ccf039725fdc077882dc2d2d5f9f987eec73561"} Jul 07 15:45:28.369628 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.369599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:28.675652 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.675513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:28.833059 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:28.833010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:29.371841 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:29.371806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2kbdn" event={"ID":"401d0f3f-db12-451d-8452-d3348ccd8de9","Type":"ContainerStarted","Data":"03f275dabc07f0159d075947f508e53ad424460857be48afcb9ba850c0c3de72"} Jul 07 15:45:30.324010 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.323975 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:30.377445 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.377348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2kbdn" event={"ID":"401d0f3f-db12-451d-8452-d3348ccd8de9","Type":"ContainerStarted","Data":"d6fb0495ab18ef70906169253e0c0ef0af000ee7ca5d11cea7869fb4a910940a"} Jul 07 15:45:30.377445 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.377430 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2kbdn" Jul 07 15:45:30.381680 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.381654 2562 generic.go:356] "Generic (PLEG): container finished" podID="6b8ce45a-0231-4117-b498-559ea6fef53a" containerID="94d911940f619ca36358e99fb46fe4bb800392ba09d20e28c0ed5c7d02ccdc81" exitCode=0 Jul 07 15:45:30.381809 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.381692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srf2g" event={"ID":"6b8ce45a-0231-4117-b498-559ea6fef53a","Type":"ContainerDied","Data":"94d911940f619ca36358e99fb46fe4bb800392ba09d20e28c0ed5c7d02ccdc81"} Jul 07 15:45:30.411627 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.408189 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2kbdn" podStartSLOduration=66.35666461 podStartE2EDuration="1m8.408170708s" totalImagesPullingTime="2.051506098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:30.406584939 +0000 UTC m=+103.637651206" watchObservedRunningTime="2026-07-07 15:45:30.408170708 +0000 UTC m=+103.639236985" Jul 07 15:45:30.439539 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.439506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:30.783641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:30.783600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:31.387732 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:31.387689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srf2g" event={"ID":"6b8ce45a-0231-4117-b498-559ea6fef53a","Type":"ContainerStarted","Data":"5c61aaee12949f7d4d9aafac5bc2bc0304a329dd418127ad1cfc773fd1055dc9"} Jul 07 15:45:31.388171 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:31.387782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srf2g" event={"ID":"6b8ce45a-0231-4117-b498-559ea6fef53a","Type":"ContainerStarted","Data":"bca8f21e940b42214ef9c9a21f7d7a15d7d280fc663bb52df15b6b2236e11058"} Jul 07 15:45:31.422365 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:31.422318 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-srf2g" podStartSLOduration=2.660076251 podStartE2EDuration="4.422301553s" totalImagesPullingTime="1.654999102s" totalInitContainerRuntime="107.2262ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:31.41998003 +0000 UTC m=+104.651046310" watchObservedRunningTime="2026-07-07 15:45:31.422301553 +0000 UTC m=+104.653367829" Jul 07 15:45:31.948577 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:31.948538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:32.212666 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:32.212555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:32.307875 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:32.307848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:33.646560 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:33.646526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:33.921117 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:33.921033 2562 scope.go:117] "RemoveContainer" containerID="de97d295b78c91b7142562e7f775d8f2db077503a52cb48c199447dea552c529" Jul 07 15:45:34.401596 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.401568 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:45:34.401788 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.401634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" event={"ID":"f8286691-e614-41cc-bffe-79ed7194bfab","Type":"ContainerStarted","Data":"88580d7bb57f8758732d2086a0820ed9b267f2bb419c14204343dc10c6d6b13a"} Jul 07 15:45:34.401919 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.401893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:45:34.420841 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.420789 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" podStartSLOduration=49.30063031 podStartE2EDuration="54.42077392s" totalImagesPullingTime="5.12014361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:34.420434415 +0000 UTC m=+107.651500711" watchObservedRunningTime="2026-07-07 15:45:34.42077392 +0000 UTC m=+107.651840194" Jul 07 15:45:34.595473 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.595440 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-d96q5" Jul 07 15:45:34.774178 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.774086 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-c75df"] Jul 07 15:45:34.777628 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.777606 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:34.780244 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.780217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 15:45:34.780361 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.780277 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7jdqt\"" Jul 07 15:45:34.780361 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.780311 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 15:45:34.788084 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.788064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-c75df"] Jul 07 15:45:34.806320 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.806296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:34.836293 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.836264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7f84\" (UniqueName: \"kubernetes.io/projected/6cf82bc3-6c92-481c-99b1-0ea546d0c5cb-kube-api-access-r7f84\") pod \"downloads-685bfb5f64-c75df\" (UID: \"6cf82bc3-6c92-481c-99b1-0ea546d0c5cb\") " pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:34.903174 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.903140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:34.936828 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.936790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7f84\" (UniqueName: \"kubernetes.io/projected/6cf82bc3-6c92-481c-99b1-0ea546d0c5cb-kube-api-access-r7f84\") pod \"downloads-685bfb5f64-c75df\" (UID: \"6cf82bc3-6c92-481c-99b1-0ea546d0c5cb\") " pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:34.945769 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:34.945746 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7f84\" (UniqueName: \"kubernetes.io/projected/6cf82bc3-6c92-481c-99b1-0ea546d0c5cb-kube-api-access-r7f84\") pod \"downloads-685bfb5f64-c75df\" (UID: \"6cf82bc3-6c92-481c-99b1-0ea546d0c5cb\") " pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:35.003210 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.003176 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:35.087333 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.087296 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:35.103964 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.103936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:35.174535 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.174501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:35.216963 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.216774 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-c75df"] Jul 07 15:45:35.219570 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:35.219532 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf82bc3_6c92_481c_99b1_0ea546d0c5cb.slice/crio-02b40a9f2409e18e167c51ce44cd9da922eea934376d30a9374b0c38e7a063bf WatchSource:0}: Error finding container 02b40a9f2409e18e167c51ce44cd9da922eea934376d30a9374b0c38e7a063bf: Status 404 returned error can't find the container with id 02b40a9f2409e18e167c51ce44cd9da922eea934376d30a9374b0c38e7a063bf Jul 07 15:45:35.406888 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.406802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-c75df" event={"ID":"6cf82bc3-6c92-481c-99b1-0ea546d0c5cb","Type":"ContainerStarted","Data":"02b40a9f2409e18e167c51ce44cd9da922eea934376d30a9374b0c38e7a063bf"} Jul 07 15:45:35.455161 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:35.455131 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:36.270036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:36.270005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:36.468036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:36.468002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:36.662207 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:36.662179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:36.878446 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:36.878413 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:45:36.966611 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:36.966532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:37.065803 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:37.065770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:37.166708 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:37.166670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:37.266181 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:37.266090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:39.808908 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:39.808853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:39.870184 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:39.870147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:40.390445 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:40.390417 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2kbdn" Jul 07 15:45:42.789443 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:42.789399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:42.892121 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:42.892083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:44.147888 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.147858 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:44.249939 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.249901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:44.692239 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.692201 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:45:44.703170 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.703146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.707511 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 15:45:44.707511 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707424 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 15:45:44.707511 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 15:45:44.707818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707419 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 15:45:44.707818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707702 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fjcp2\"" Jul 07 15:45:44.707818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.707730 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 15:45:44.708118 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.708093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:45:44.823774 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.823953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.823953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.823953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.823953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.824165 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.823977 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq6jj\" (UniqueName: \"kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925291 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925496 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925496 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925496 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925496 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq6jj\" (UniqueName: \"kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.925698 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.925574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.926144 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.926095 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.926244 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.926207 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.926244 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.926207 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.928223 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.928197 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.928327 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.928229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:44.934880 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:44.934848 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq6jj\" (UniqueName: \"kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj\") pod \"console-6866bb9575-clvv4\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:45.015440 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:45.015339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:45:45.574613 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:45.574582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:45.678303 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:45.678268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:46.883697 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:46.883663 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:45:49.288032 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:49.287999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:50.573125 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:50.573082 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:50.670660 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:50.670625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:51.960188 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:51.960158 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:45:51.963450 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:51.963426 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a64fd9_c908_433a_9f2a_e62d5ef1003c.slice/crio-fb514b2142094f45bfa8d5ba00ea2a32c297b3b1411ad0cbd634d42794012ef9 WatchSource:0}: Error finding container fb514b2142094f45bfa8d5ba00ea2a32c297b3b1411ad0cbd634d42794012ef9: Status 404 returned error can't find the container with id fb514b2142094f45bfa8d5ba00ea2a32c297b3b1411ad0cbd634d42794012ef9 Jul 07 15:45:52.477355 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:52.477301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-c75df" event={"ID":"6cf82bc3-6c92-481c-99b1-0ea546d0c5cb","Type":"ContainerStarted","Data":"dccb340848c9ce17023bc8f3cf0079f303803748dcf8154704b21452f160e56a"} Jul 07 15:45:52.477586 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:52.477504 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:52.478936 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:52.478908 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6866bb9575-clvv4" event={"ID":"10a64fd9-c908-433a-9f2a-e62d5ef1003c","Type":"ContainerStarted","Data":"fb514b2142094f45bfa8d5ba00ea2a32c297b3b1411ad0cbd634d42794012ef9"} Jul 07 15:45:52.491036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:52.491012 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-c75df" Jul 07 15:45:52.504061 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:52.504008 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-c75df" podStartSLOduration=1.804444655 podStartE2EDuration="18.503988824s" totalImagesPullingTime="16.699544169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:52.502418145 +0000 UTC m=+125.733484441" watchObservedRunningTime="2026-07-07 15:45:52.503988824 +0000 UTC m=+125.735055101" Jul 07 15:45:53.296943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.296896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:53.397274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.397232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:53.487965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.484855 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:45:53.491164 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.491132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.501198 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.501165 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:45:53.506214 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.506189 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 15:45:53.605959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.605850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.605959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.605910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.606641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.605991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.606641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.606085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.606641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.606142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.606641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.606203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmscs\" (UniqueName: \"kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.606641 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.606231 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmscs\" (UniqueName: \"kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707782 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.707887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.709031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.709653 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.709449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.710244 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.710202 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.711035 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.710970 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.712356 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.712331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.715195 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.715035 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.722484 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.722438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmscs\" (UniqueName: \"kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs\") pod \"console-6d985f7dc6-xx29r\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.807495 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.807455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:45:53.985686 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:53.985546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:45:53.988802 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:53.988755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4b67c4_169c_4de4_bf1f_ecbce9632bbd.slice/crio-36046e5b765b6128294cc8609315ac1d043ea12f23e44fb41dd0385977ac7fb3 WatchSource:0}: Error finding container 36046e5b765b6128294cc8609315ac1d043ea12f23e44fb41dd0385977ac7fb3: Status 404 returned error can't find the container with id 36046e5b765b6128294cc8609315ac1d043ea12f23e44fb41dd0385977ac7fb3 Jul 07 15:45:54.499061 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:54.499020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d985f7dc6-xx29r" event={"ID":"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd","Type":"ContainerStarted","Data":"36046e5b765b6128294cc8609315ac1d043ea12f23e44fb41dd0385977ac7fb3"} Jul 07 15:45:54.581457 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:54.581416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:54.658701 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:54.658583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:55.098263 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:55.098223 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:55.168735 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:55.168666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:45:56.510242 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:56.509622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d985f7dc6-xx29r" event={"ID":"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd","Type":"ContainerStarted","Data":"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240"} Jul 07 15:45:56.511435 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:56.511406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6866bb9575-clvv4" event={"ID":"10a64fd9-c908-433a-9f2a-e62d5ef1003c","Type":"ContainerStarted","Data":"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959"} Jul 07 15:45:56.531175 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:56.531118 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d985f7dc6-xx29r" podStartSLOduration=1.384895733 podStartE2EDuration="3.531097863s" totalImagesPullingTime="2.14620213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:56.52912482 +0000 UTC m=+129.760191112" watchObservedRunningTime="2026-07-07 15:45:56.531097863 +0000 UTC m=+129.762164139" Jul 07 15:45:56.547600 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:56.547549 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6866bb9575-clvv4" podStartSLOduration=8.376238529 podStartE2EDuration="12.547533846s" totalImagesPullingTime="4.171295317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:56.5467155 +0000 UTC m=+129.777781777" watchObservedRunningTime="2026-07-07 15:45:56.547533846 +0000 UTC m=+129.778600121" Jul 07 15:45:57.250583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:57.250538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:45:57.253342 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:57.253308 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639ebdf2-6bbe-42a5-8ade-322a278cad2d-metrics-certs\") pod \"network-metrics-daemon-42jm9\" (UID: \"639ebdf2-6bbe-42a5-8ade-322a278cad2d\") " pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:45:57.449163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:57.449129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wcvdn\"" Jul 07 15:45:57.456593 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:57.456560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42jm9" Jul 07 15:45:57.640245 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:57.640214 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-42jm9"] Jul 07 15:45:57.643671 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:45:57.643638 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639ebdf2_6bbe_42a5_8ade_322a278cad2d.slice/crio-2e7c8c0c16f5055433d441a00c7fd85602b5700ccc308df578b77f7ca339aa55 WatchSource:0}: Error finding container 2e7c8c0c16f5055433d441a00c7fd85602b5700ccc308df578b77f7ca339aa55: Status 404 returned error can't find the container with id 2e7c8c0c16f5055433d441a00c7fd85602b5700ccc308df578b77f7ca339aa55 Jul 07 15:45:58.523670 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:45:58.523603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42jm9" event={"ID":"639ebdf2-6bbe-42a5-8ade-322a278cad2d","Type":"ContainerStarted","Data":"2e7c8c0c16f5055433d441a00c7fd85602b5700ccc308df578b77f7ca339aa55"} Jul 07 15:46:00.534817 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:00.534782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42jm9" event={"ID":"639ebdf2-6bbe-42a5-8ade-322a278cad2d","Type":"ContainerStarted","Data":"f6121c58c38f0a7980ca9468a19d95202625af3d792858bc4a735a0311397cfa"} Jul 07 15:46:00.535263 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:00.534844 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42jm9" event={"ID":"639ebdf2-6bbe-42a5-8ade-322a278cad2d","Type":"ContainerStarted","Data":"fedefe73739ab67aa894bd123121d6cbbb9f0ede35564846e5c694541d3a5269"} Jul 07 15:46:00.554127 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:00.554039 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-42jm9" podStartSLOduration=130.79574003 podStartE2EDuration="2m12.554017747s" totalImagesPullingTime="1.758277717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:00.552701274 +0000 UTC m=+133.783767573" watchObservedRunningTime="2026-07-07 15:46:00.554017747 +0000 UTC m=+133.785084023" Jul 07 15:46:00.682209 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:00.682175 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:00.770426 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:00.770393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:01.909083 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:01.909033 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" podUID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" containerName="registry" containerID="cri-o://706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7" gracePeriod=30 Jul 07 15:46:02.190160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.190121 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:46:02.295680 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295703 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295737 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295767 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295782 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295802 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzw8w\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.295923 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.295865 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca\") pod \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\" (UID: \"e463e2ad-2e40-4669-b119-f0f6e69f6f8b\") " Jul 07 15:46:02.296273 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.296124 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:02.296528 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.296486 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:02.298657 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.298623 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:02.298767 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.298620 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w" (OuterVolumeSpecName: "kube-api-access-bzw8w") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "kube-api-access-bzw8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:02.298767 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.298737 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:02.298840 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.298778 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:02.298840 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.298792 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:02.304812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.304786 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e463e2ad-2e40-4669-b119-f0f6e69f6f8b" (UID: "e463e2ad-2e40-4669-b119-f0f6e69f6f8b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:46:02.397092 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397049 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-ca-trust-extracted\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397092 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397082 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-tls\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397092 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397092 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-installation-pull-secrets\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397092 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397101 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-bound-sa-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397374 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397109 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzw8w\" (UniqueName: \"kubernetes.io/projected/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-kube-api-access-bzw8w\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397374 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397117 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-trusted-ca\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397374 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397126 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-registry-certificates\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.397374 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.397134 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e463e2ad-2e40-4669-b119-f0f6e69f6f8b-image-registry-private-configuration\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:02.543988 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.543950 2562 generic.go:356] "Generic (PLEG): container finished" podID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" containerID="706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7" exitCode=0 Jul 07 15:46:02.544160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.543994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" event={"ID":"e463e2ad-2e40-4669-b119-f0f6e69f6f8b","Type":"ContainerDied","Data":"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7"} Jul 07 15:46:02.544160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.543998 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" Jul 07 15:46:02.544160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.544038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b666b688f-8cm8m" event={"ID":"e463e2ad-2e40-4669-b119-f0f6e69f6f8b","Type":"ContainerDied","Data":"b0c35f2f1b02e6ded55197cc31e45b36016cc8e8ebc0203a0d538f949e75a3f9"} Jul 07 15:46:02.544160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.544060 2562 scope.go:117] "RemoveContainer" containerID="706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7" Jul 07 15:46:02.552648 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.552631 2562 scope.go:117] "RemoveContainer" containerID="706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7" Jul 07 15:46:02.552953 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:46:02.552932 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7\": container with ID starting with 706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7 not found: ID does not exist" containerID="706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7" Jul 07 15:46:02.552991 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.552963 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7"} err="failed to get container status \"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7\": rpc error: code = NotFound desc = could not find container \"706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7\": container with ID starting with 706b116650deb7017e6beaaf9e176999af5e3b225d9ccd9ee38fb71b87b4b1e7 not found: ID does not exist" Jul 07 15:46:02.565160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.565119 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:46:02.568956 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:02.568927 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b666b688f-8cm8m"] Jul 07 15:46:03.461644 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.461609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:03.537459 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.537427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:03.808526 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.808484 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:46:03.808731 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.808557 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:46:03.813398 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.813345 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:46:03.925238 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:03.925207 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" path="/var/lib/kubelet/pods/e463e2ad-2e40-4669-b119-f0f6e69f6f8b/volumes" Jul 07 15:46:04.555462 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:04.555434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:46:04.603128 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:04.603098 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:46:04.713895 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:04.713864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:04.780021 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:04.779985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:04.919442 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:04.919348 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:05.015617 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:05.015584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:46:05.020262 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:05.020234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:05.574856 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:05.574822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:05.639349 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:05.639315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:07.564661 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:07.564627 2562 generic.go:356] "Generic (PLEG): container finished" podID="f702084c-90f6-49db-a2fd-5a5bee9e76be" containerID="def465cea29318e69212a4ebca47261d8e65976c2113c83ccf51b113c9b682fb" exitCode=0 Jul 07 15:46:07.565131 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:07.564707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" event={"ID":"f702084c-90f6-49db-a2fd-5a5bee9e76be","Type":"ContainerDied","Data":"def465cea29318e69212a4ebca47261d8e65976c2113c83ccf51b113c9b682fb"} Jul 07 15:46:07.565131 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:07.565120 2562 scope.go:117] "RemoveContainer" containerID="def465cea29318e69212a4ebca47261d8e65976c2113c83ccf51b113c9b682fb" Jul 07 15:46:08.570133 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:08.570096 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bvhfr" event={"ID":"f702084c-90f6-49db-a2fd-5a5bee9e76be","Type":"ContainerStarted","Data":"91d168f879601f3652401a424fafa83d95c0417bb683554ca0c36b4c834b5da6"} Jul 07 15:46:16.600492 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:16.600457 2562 generic.go:356] "Generic (PLEG): container finished" podID="cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648" containerID="09691c5da7a55b5a653139734507453eda6a13877d0bbe4e5f54847df83ee10f" exitCode=0 Jul 07 15:46:16.600868 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:16.600539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" event={"ID":"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648","Type":"ContainerDied","Data":"09691c5da7a55b5a653139734507453eda6a13877d0bbe4e5f54847df83ee10f"} Jul 07 15:46:16.600915 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:16.600878 2562 scope.go:117] "RemoveContainer" containerID="09691c5da7a55b5a653139734507453eda6a13877d0bbe4e5f54847df83ee10f" Jul 07 15:46:17.605657 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:17.605622 2562 generic.go:356] "Generic (PLEG): container finished" podID="31228800-77a6-4869-9090-7cbe6fa30887" containerID="ec2e32046931fbc0c68e5809efcd705841a0a2bdc88ea90ff0193e918a0140c1" exitCode=0 Jul 07 15:46:17.606089 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:17.605694 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" event={"ID":"31228800-77a6-4869-9090-7cbe6fa30887","Type":"ContainerDied","Data":"ec2e32046931fbc0c68e5809efcd705841a0a2bdc88ea90ff0193e918a0140c1"} Jul 07 15:46:17.606089 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:17.606060 2562 scope.go:117] "RemoveContainer" containerID="ec2e32046931fbc0c68e5809efcd705841a0a2bdc88ea90ff0193e918a0140c1" Jul 07 15:46:17.607422 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:17.607404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q8xlf" event={"ID":"cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648","Type":"ContainerStarted","Data":"41a4e76d81a069c88e63605e385f234c04fb261f0399f3cb2dab67313ae794d9"} Jul 07 15:46:18.612992 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:18.612956 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-mn4xv" event={"ID":"31228800-77a6-4869-9090-7cbe6fa30887","Type":"ContainerStarted","Data":"b28b929c217be2dfdd5b0d3139546b2073d85759a9c679a6d105f6e6f7405cc5"} Jul 07 15:46:27.784342 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:27.784305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:27.889130 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:27.889090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:29.632826 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:29.632778 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6866bb9575-clvv4" podUID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" containerName="console" containerID="cri-o://8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959" gracePeriod=15 Jul 07 15:46:29.920046 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:29.920013 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:46:30.018545 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018504 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.018545 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018556 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq6jj\" (UniqueName: \"kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.018812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018579 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.018812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018599 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.018812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018658 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.018812 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.018760 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert\") pod \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\" (UID: \"10a64fd9-c908-433a-9f2a-e62d5ef1003c\") " Jul 07 15:46:30.019091 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.019062 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca" (OuterVolumeSpecName: "service-ca") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:30.019220 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.019086 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config" (OuterVolumeSpecName: "console-config") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:30.019220 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.019194 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:30.020962 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.020935 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:30.021054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.020971 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:30.021054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.020985 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj" (OuterVolumeSpecName: "kube-api-access-mq6jj") pod "10a64fd9-c908-433a-9f2a-e62d5ef1003c" (UID: "10a64fd9-c908-433a-9f2a-e62d5ef1003c"). InnerVolumeSpecName "kube-api-access-mq6jj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:30.119571 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119533 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-oauth-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.119571 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119562 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq6jj\" (UniqueName: \"kubernetes.io/projected/10a64fd9-c908-433a-9f2a-e62d5ef1003c-kube-api-access-mq6jj\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.119571 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119572 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-service-ca\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.119571 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119584 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.119829 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119592 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a64fd9-c908-433a-9f2a-e62d5ef1003c-console-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.119829 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.119600 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a64fd9-c908-433a-9f2a-e62d5ef1003c-oauth-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:46:30.660541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.660502 2562 generic.go:356] "Generic (PLEG): container finished" podID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" containerID="8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959" exitCode=0 Jul 07 15:46:30.660938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.660551 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6866bb9575-clvv4" Jul 07 15:46:30.660938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.660585 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6866bb9575-clvv4" event={"ID":"10a64fd9-c908-433a-9f2a-e62d5ef1003c","Type":"ContainerDied","Data":"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959"} Jul 07 15:46:30.660938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.660637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6866bb9575-clvv4" event={"ID":"10a64fd9-c908-433a-9f2a-e62d5ef1003c","Type":"ContainerDied","Data":"fb514b2142094f45bfa8d5ba00ea2a32c297b3b1411ad0cbd634d42794012ef9"} Jul 07 15:46:30.660938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.660657 2562 scope.go:117] "RemoveContainer" containerID="8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959" Jul 07 15:46:30.668937 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.668919 2562 scope.go:117] "RemoveContainer" containerID="8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959" Jul 07 15:46:30.669226 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:46:30.669205 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959\": container with ID starting with 8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959 not found: ID does not exist" containerID="8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959" Jul 07 15:46:30.669271 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.669235 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959"} err="failed to get container status \"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959\": rpc error: code = NotFound desc = could not find container \"8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959\": container with ID starting with 8ee4a505b49ec2af02996468f682822acd51b30aa49760e9dd238750d2929959 not found: ID does not exist" Jul 07 15:46:30.682137 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.682102 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:46:30.686304 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:30.686283 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6866bb9575-clvv4"] Jul 07 15:46:31.927338 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:31.927309 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" path="/var/lib/kubelet/pods/10a64fd9-c908-433a-9f2a-e62d5ef1003c/volumes" Jul 07 15:46:39.138054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:39.138004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:43.481028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:43.480989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:43.582222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:43.582178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:52.060115 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060077 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:46:52.060502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060399 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" containerName="console" Jul 07 15:46:52.060502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060412 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" containerName="console" Jul 07 15:46:52.060502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060431 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" containerName="registry" Jul 07 15:46:52.060502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060467 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" containerName="registry" Jul 07 15:46:52.060625 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060516 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a64fd9-c908-433a-9f2a-e62d5ef1003c" containerName="console" Jul 07 15:46:52.060625 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.060525 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e463e2ad-2e40-4669-b119-f0f6e69f6f8b" containerName="registry" Jul 07 15:46:52.064907 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.064882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.075659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.075623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:46:52.116004 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.115942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:52.200288 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200288 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.200544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.200448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x726h\" (UniqueName: \"kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.264818 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.264767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:52.301455 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301455 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.301929 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.301748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x726h\" (UniqueName: \"kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.302359 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.302337 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.302359 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.302354 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.302541 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.302362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.302803 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.302780 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.304113 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.304086 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.304489 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.304472 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.312506 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.312446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x726h\" (UniqueName: \"kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h\") pod \"console-6859fbc6c9-d6bt7\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.375222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.375175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:46:52.508943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.508912 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:46:52.511263 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:46:52.511233 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205c3cea_c3cf_4273_bd06_4b8f5233f52e.slice/crio-b2d634e3f0a6fb9023243c4d49a96d1b30c2fc82ea37fff62ddddc8fafc0b18d WatchSource:0}: Error finding container b2d634e3f0a6fb9023243c4d49a96d1b30c2fc82ea37fff62ddddc8fafc0b18d: Status 404 returned error can't find the container with id b2d634e3f0a6fb9023243c4d49a96d1b30c2fc82ea37fff62ddddc8fafc0b18d Jul 07 15:46:52.745354 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.745311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859fbc6c9-d6bt7" event={"ID":"205c3cea-c3cf-4273-bd06-4b8f5233f52e","Type":"ContainerStarted","Data":"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6"} Jul 07 15:46:52.745354 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.745366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859fbc6c9-d6bt7" event={"ID":"205c3cea-c3cf-4273-bd06-4b8f5233f52e","Type":"ContainerStarted","Data":"b2d634e3f0a6fb9023243c4d49a96d1b30c2fc82ea37fff62ddddc8fafc0b18d"} Jul 07 15:46:52.765821 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:52.765767 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6859fbc6c9-d6bt7" podStartSLOduration=0.765750341 podStartE2EDuration="765.750341ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:52.764594698 +0000 UTC m=+185.995660992" watchObservedRunningTime="2026-07-07 15:46:52.765750341 +0000 UTC m=+185.996816616" Jul 07 15:46:55.378182 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:55.378147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:46:55.452925 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:46:55.452895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:02.376086 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.376012 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:47:02.376603 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.376122 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:47:02.381121 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.381093 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:47:02.785371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.785275 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:47:02.838646 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.838607 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:47:02.951977 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:02.951938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:03.052873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:03.052836 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:03.119175 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:03.119139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:12.773956 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:12.773912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:12.911599 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:12.911564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:27.861509 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:27.861460 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d985f7dc6-xx29r" podUID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" containerName="console" containerID="cri-o://0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240" gracePeriod=15 Jul 07 15:47:28.104544 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.104519 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:47:28.192693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192596 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192637 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmscs\" (UniqueName: \"kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192684 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192717 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192767 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.192965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.192820 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle\") pod \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\" (UID: \"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd\") " Jul 07 15:47:28.193112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.193058 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:47:28.193236 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.193210 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca" (OuterVolumeSpecName: "service-ca") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:47:28.193322 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.193266 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config" (OuterVolumeSpecName: "console-config") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:47:28.193420 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.193398 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:47:28.194954 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.194926 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:47:28.195317 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.195289 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:47:28.195317 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.195304 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs" (OuterVolumeSpecName: "kube-api-access-qmscs") pod "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" (UID: "7b4b67c4-169c-4de4-bf1f-ecbce9632bbd"). InnerVolumeSpecName "kube-api-access-qmscs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:47:28.294028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.293975 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294023 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-trusted-ca-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294033 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294028 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294042 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmscs\" (UniqueName: \"kubernetes.io/projected/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-kube-api-access-qmscs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294051 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-service-ca\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294060 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-oauth-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.294300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.294068 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd-console-oauth-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:47:28.880853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.880818 2562 generic.go:356] "Generic (PLEG): container finished" podID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" containerID="0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240" exitCode=0 Jul 07 15:47:28.881283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.880862 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d985f7dc6-xx29r" Jul 07 15:47:28.881283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.880868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d985f7dc6-xx29r" event={"ID":"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd","Type":"ContainerDied","Data":"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240"} Jul 07 15:47:28.881283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.880966 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d985f7dc6-xx29r" event={"ID":"7b4b67c4-169c-4de4-bf1f-ecbce9632bbd","Type":"ContainerDied","Data":"36046e5b765b6128294cc8609315ac1d043ea12f23e44fb41dd0385977ac7fb3"} Jul 07 15:47:28.881283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.880987 2562 scope.go:117] "RemoveContainer" containerID="0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240" Jul 07 15:47:28.889284 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.889265 2562 scope.go:117] "RemoveContainer" containerID="0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240" Jul 07 15:47:28.889555 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:47:28.889535 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240\": container with ID starting with 0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240 not found: ID does not exist" containerID="0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240" Jul 07 15:47:28.889628 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.889566 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240"} err="failed to get container status \"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240\": rpc error: code = NotFound desc = could not find container \"0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240\": container with ID starting with 0c1062638dda80fbe377cf6ee4ef6bc4eee7964c8e507289835dc3afe1401240 not found: ID does not exist" Jul 07 15:47:28.902037 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.902013 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:47:28.905389 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:28.905356 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d985f7dc6-xx29r"] Jul 07 15:47:29.925271 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:29.925237 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" path="/var/lib/kubelet/pods/7b4b67c4-169c-4de4-bf1f-ecbce9632bbd/volumes" Jul 07 15:47:30.511965 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:30.511937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:34.032652 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:34.032607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:38.816716 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:38.816683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:38.911985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:38.911951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:56.972070 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:56.972041 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:47:57.069664 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:47:57.069634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:48:29.364579 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:29.364544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:48:29.466635 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:29.466583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:48:41.148529 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.148491 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk"] Jul 07 15:48:41.149129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.148908 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" containerName="console" Jul 07 15:48:41.149129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.148927 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" containerName="console" Jul 07 15:48:41.149129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.149004 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4b67c4-169c-4de4-bf1f-ecbce9632bbd" containerName="console" Jul 07 15:48:41.152150 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.152129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.155148 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.155122 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:48:41.155260 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.155152 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:48:41.155260 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.155170 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ljfx4\"" Jul 07 15:48:41.160805 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.160783 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk"] Jul 07 15:48:41.270190 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.270138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7s7n\" (UniqueName: \"kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.270364 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.270237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.270364 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.270264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.371720 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.371679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7s7n\" (UniqueName: \"kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.371918 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.371747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.371918 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.371776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.372128 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.372104 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.372203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.372179 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.379569 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.379546 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7s7n\" (UniqueName: \"kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.461214 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.461125 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:41.591448 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:41.591425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk"] Jul 07 15:48:41.593932 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:48:41.593904 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf751fd35_2f27_4e5d_878c_f5d4ad29a1ad.slice/crio-5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e WatchSource:0}: Error finding container 5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e: Status 404 returned error can't find the container with id 5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e Jul 07 15:48:42.161523 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:42.161482 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" event={"ID":"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad","Type":"ContainerStarted","Data":"5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e"} Jul 07 15:48:47.183740 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.183706 2562 generic.go:356] "Generic (PLEG): container finished" podID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerID="caf94ee4e77346047f7a1bff6f9f618d2f37f521fecc9fac605b5316bac781e4" exitCode=0 Jul 07 15:48:47.184150 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.183791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" event={"ID":"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad","Type":"ContainerDied","Data":"caf94ee4e77346047f7a1bff6f9f618d2f37f521fecc9fac605b5316bac781e4"} Jul 07 15:48:47.306704 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.306679 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:48:47.313084 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.313047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:48:47.314545 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.314526 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:48:47.318205 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.318179 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:48:47.320667 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:47.320651 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:48:49.193693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:49.193611 2562 generic.go:356] "Generic (PLEG): container finished" podID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerID="7bc5d0053161e56dffae6c351948f7f86a69c4775fd626c1d09f9d3a26b40a41" exitCode=0 Jul 07 15:48:49.194082 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:49.193695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" event={"ID":"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad","Type":"ContainerDied","Data":"7bc5d0053161e56dffae6c351948f7f86a69c4775fd626c1d09f9d3a26b40a41"} Jul 07 15:48:49.194689 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:49.194673 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:48:57.229760 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:57.229728 2562 generic.go:356] "Generic (PLEG): container finished" podID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerID="ccd6884e4087eb4a1a2275d820fb02149061d2e22ac48f5653188f07edcd5d22" exitCode=0 Jul 07 15:48:57.230122 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:57.229774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" event={"ID":"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad","Type":"ContainerDied","Data":"ccd6884e4087eb4a1a2275d820fb02149061d2e22ac48f5653188f07edcd5d22"} Jul 07 15:48:58.359877 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.359845 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:58.526326 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.526240 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util\") pod \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " Jul 07 15:48:58.526326 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.526311 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7s7n\" (UniqueName: \"kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n\") pod \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " Jul 07 15:48:58.526560 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.526337 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle\") pod \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\" (UID: \"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad\") " Jul 07 15:48:58.527088 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.527064 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle" (OuterVolumeSpecName: "bundle") pod "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" (UID: "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:48:58.528582 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.528557 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n" (OuterVolumeSpecName: "kube-api-access-b7s7n") pod "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" (UID: "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad"). InnerVolumeSpecName "kube-api-access-b7s7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:48:58.530158 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.530138 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util" (OuterVolumeSpecName: "util") pod "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" (UID: "f751fd35-2f27-4e5d-878c-f5d4ad29a1ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:48:58.627130 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.627096 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:48:58.627130 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.627125 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:48:58.627130 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:58.627135 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7s7n\" (UniqueName: \"kubernetes.io/projected/f751fd35-2f27-4e5d-878c-f5d4ad29a1ad-kube-api-access-b7s7n\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:48:59.245110 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:59.245078 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" Jul 07 15:48:59.245274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:59.245078 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540smhlk" event={"ID":"f751fd35-2f27-4e5d-878c-f5d4ad29a1ad","Type":"ContainerDied","Data":"5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e"} Jul 07 15:48:59.245274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:48:59.245228 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5811f2ef1a7b2702357a4a7bc02b187a3817a2322365b86797bd1b682cfa5d4e" Jul 07 15:49:02.161727 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:02.161697 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:02.932700 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:02.932667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:04.180106 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180072 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt"] Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180351 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="extract" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180362 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="extract" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180372 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="pull" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180390 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="pull" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180417 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="util" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180423 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="util" Jul 07 15:49:04.180510 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.180478 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f751fd35-2f27-4e5d-878c-f5d4ad29a1ad" containerName="extract" Jul 07 15:49:04.228210 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.228169 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt"] Jul 07 15:49:04.228371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.228273 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.234133 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.234104 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 15:49:04.234283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.234190 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l5hjz\"" Jul 07 15:49:04.234283 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.234202 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:49:04.374010 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.373969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxr6h\" (UniqueName: \"kubernetes.io/projected/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-kube-api-access-cxr6h\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.374212 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.374024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.474658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.474570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxr6h\" (UniqueName: \"kubernetes.io/projected/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-kube-api-access-cxr6h\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.474658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.474616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.475023 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.475007 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.487554 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.487526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxr6h\" (UniqueName: \"kubernetes.io/projected/ebde36e5-39b4-4a33-8354-bcdc369d1c6a-kube-api-access-cxr6h\") pod \"cert-manager-operator-controller-manager-55c444bf86-2wftt\" (UID: \"ebde36e5-39b4-4a33-8354-bcdc369d1c6a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.537533 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.537496 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" Jul 07 15:49:04.700852 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:04.700823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt"] Jul 07 15:49:04.703472 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:04.703449 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebde36e5_39b4_4a33_8354_bcdc369d1c6a.slice/crio-19d9b75dfe6eb6ba6d4603636b3aee856edfad3165c7d6f784655e60f0932e8c WatchSource:0}: Error finding container 19d9b75dfe6eb6ba6d4603636b3aee856edfad3165c7d6f784655e60f0932e8c: Status 404 returned error can't find the container with id 19d9b75dfe6eb6ba6d4603636b3aee856edfad3165c7d6f784655e60f0932e8c Jul 07 15:49:05.271139 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:05.271105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" event={"ID":"ebde36e5-39b4-4a33-8354-bcdc369d1c6a","Type":"ContainerStarted","Data":"19d9b75dfe6eb6ba6d4603636b3aee856edfad3165c7d6f784655e60f0932e8c"} Jul 07 15:49:05.751543 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:05.751504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:05.851351 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:05.851316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:07.284355 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.284321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" event={"ID":"ebde36e5-39b4-4a33-8354-bcdc369d1c6a","Type":"ContainerStarted","Data":"9fea97187fc916fa10d9d06c77f30d1c5854c6a3432d3cd81797239017f1a933"} Jul 07 15:49:07.308390 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.308334 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2wftt" podStartSLOduration=1.288043637 podStartE2EDuration="3.308315212s" totalImagesPullingTime="2.020271575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:07.306872612 +0000 UTC m=+320.537938886" watchObservedRunningTime="2026-07-07 15:49:07.308315212 +0000 UTC m=+320.539381490" Jul 07 15:49:07.442049 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.442013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:07.569123 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.569087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:07.695123 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.695079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:07.809200 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:07.809165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:08.391261 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.391225 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6"] Jul 07 15:49:08.415478 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.415445 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6"] Jul 07 15:49:08.415636 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.415551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.418821 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.418792 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:49:08.418960 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.418896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ljfx4\"" Jul 07 15:49:08.418960 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.418943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:49:08.509221 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.509183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.509418 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.509232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpgnl\" (UniqueName: \"kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.509418 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.509333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.610643 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.610600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.610814 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.610667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.610814 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.610691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpgnl\" (UniqueName: \"kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.611054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.611029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.611112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.611049 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.619504 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.619475 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpgnl\" (UniqueName: \"kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.726032 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.725937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:08.898103 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:08.898078 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6"] Jul 07 15:49:08.900717 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:08.900689 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0b9a07_95f5_4684_b967_9dd7956f5923.slice/crio-fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1 WatchSource:0}: Error finding container fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1: Status 404 returned error can't find the container with id fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1 Jul 07 15:49:09.293361 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:09.293326 2562 generic.go:356] "Generic (PLEG): container finished" podID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerID="cb3e000a93260201faad7f3329d087ede44bdd7f0d68d82f969b3cd03961146f" exitCode=0 Jul 07 15:49:09.293590 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:09.293369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" event={"ID":"fc0b9a07-95f5-4684-b967-9dd7956f5923","Type":"ContainerDied","Data":"cb3e000a93260201faad7f3329d087ede44bdd7f0d68d82f969b3cd03961146f"} Jul 07 15:49:09.293590 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:09.293429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" event={"ID":"fc0b9a07-95f5-4684-b967-9dd7956f5923","Type":"ContainerStarted","Data":"fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1"} Jul 07 15:49:09.855163 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:09.855126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:11.946451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:11.946416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:12.308184 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:12.308145 2562 generic.go:356] "Generic (PLEG): container finished" podID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerID="74adbbda3ddfcca913c2dc406b8811c14ea0b48f86c1b94b6e333a037cb9e40f" exitCode=0 Jul 07 15:49:12.308347 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:12.308231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" event={"ID":"fc0b9a07-95f5-4684-b967-9dd7956f5923","Type":"ContainerDied","Data":"74adbbda3ddfcca913c2dc406b8811c14ea0b48f86c1b94b6e333a037cb9e40f"} Jul 07 15:49:12.567646 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:12.567560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:12.662645 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:12.662618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:13.314602 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:13.314572 2562 generic.go:356] "Generic (PLEG): container finished" podID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerID="f603811ed7a9a3abe1d5fd2729724dabd1bbc746f0387ab887187fdbb2e104aa" exitCode=0 Jul 07 15:49:13.314991 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:13.314615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" event={"ID":"fc0b9a07-95f5-4684-b967-9dd7956f5923","Type":"ContainerDied","Data":"f603811ed7a9a3abe1d5fd2729724dabd1bbc746f0387ab887187fdbb2e104aa"} Jul 07 15:49:14.425832 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.425796 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kv4hx"] Jul 07 15:49:14.430059 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.430037 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.432732 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.432711 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 15:49:14.432831 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.432764 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 15:49:14.432831 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.432804 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8cjt4\"" Jul 07 15:49:14.440280 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.440258 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kv4hx"] Jul 07 15:49:14.479793 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.479766 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:14.560848 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.560813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.560848 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.560847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbh4c\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-kube-api-access-zbh4c\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.661363 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661276 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle\") pod \"fc0b9a07-95f5-4684-b967-9dd7956f5923\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " Jul 07 15:49:14.661363 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661326 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util\") pod \"fc0b9a07-95f5-4684-b967-9dd7956f5923\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " Jul 07 15:49:14.661363 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661360 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpgnl\" (UniqueName: \"kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl\") pod \"fc0b9a07-95f5-4684-b967-9dd7956f5923\" (UID: \"fc0b9a07-95f5-4684-b967-9dd7956f5923\") " Jul 07 15:49:14.661608 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.661608 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbh4c\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-kube-api-access-zbh4c\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.661766 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.661730 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle" (OuterVolumeSpecName: "bundle") pod "fc0b9a07-95f5-4684-b967-9dd7956f5923" (UID: "fc0b9a07-95f5-4684-b967-9dd7956f5923"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:49:14.663658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.663628 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl" (OuterVolumeSpecName: "kube-api-access-fpgnl") pod "fc0b9a07-95f5-4684-b967-9dd7956f5923" (UID: "fc0b9a07-95f5-4684-b967-9dd7956f5923"). InnerVolumeSpecName "kube-api-access-fpgnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:49:14.666079 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.666043 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util" (OuterVolumeSpecName: "util") pod "fc0b9a07-95f5-4684-b967-9dd7956f5923" (UID: "fc0b9a07-95f5-4684-b967-9dd7956f5923"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:49:14.670322 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.670293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.670681 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.670658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbh4c\" (UniqueName: \"kubernetes.io/projected/ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a-kube-api-access-zbh4c\") pod \"cert-manager-webhook-6486ffd997-kv4hx\" (UID: \"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.762725 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.762689 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpgnl\" (UniqueName: \"kubernetes.io/projected/fc0b9a07-95f5-4684-b967-9dd7956f5923-kube-api-access-fpgnl\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:14.762725 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.762722 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:14.762725 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.762735 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0b9a07-95f5-4684-b967-9dd7956f5923-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:14.773907 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.773886 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:14.904444 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:14.904415 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kv4hx"] Jul 07 15:49:14.907069 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:14.907040 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac4f26c7_5bef_40c1_8ef4_7b1f2c182a0a.slice/crio-710b6d44c9b162c9e28699aaf07faba449a13a0415c5329e3976ca214f0c166b WatchSource:0}: Error finding container 710b6d44c9b162c9e28699aaf07faba449a13a0415c5329e3976ca214f0c166b: Status 404 returned error can't find the container with id 710b6d44c9b162c9e28699aaf07faba449a13a0415c5329e3976ca214f0c166b Jul 07 15:49:15.324666 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:15.324641 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" Jul 07 15:49:15.324837 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:15.324634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwrb6" event={"ID":"fc0b9a07-95f5-4684-b967-9dd7956f5923","Type":"ContainerDied","Data":"fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1"} Jul 07 15:49:15.324837 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:15.324758 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fed48217f11260557add584d925f05db4f198366001bb592e9e56e86e9bb4dd1" Jul 07 15:49:15.325873 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:15.325851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" event={"ID":"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a","Type":"ContainerStarted","Data":"710b6d44c9b162c9e28699aaf07faba449a13a0415c5329e3976ca214f0c166b"} Jul 07 15:49:18.056932 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:18.056884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:18.340981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:18.340884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" event={"ID":"ac4f26c7-5bef-40c1-8ef4-7b1f2c182a0a","Type":"ContainerStarted","Data":"4ec341d1d8750dbac7ffdf43080453a8edc87ee901088fc5b3ca5e0c2febc68b"} Jul 07 15:49:18.341158 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:18.341006 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:18.364036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:18.363969 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" podStartSLOduration=1.906502094 podStartE2EDuration="4.363952006s" totalImagesPullingTime="2.457449912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:18.363228992 +0000 UTC m=+331.594295270" watchObservedRunningTime="2026-07-07 15:49:18.363952006 +0000 UTC m=+331.595018280" Jul 07 15:49:18.951931 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:18.951898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:19.052944 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:19.052907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:20.313728 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.313691 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hzp4g"] Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.313981 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="extract" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.313993 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="extract" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.314008 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="util" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.314013 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="util" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.314020 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="pull" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.314025 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="pull" Jul 07 15:49:20.314112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.314075 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0b9a07-95f5-4684-b967-9dd7956f5923" containerName="extract" Jul 07 15:49:20.317051 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.317035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.319617 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.319598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7q7xd\"" Jul 07 15:49:20.327548 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.327525 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hzp4g"] Jul 07 15:49:20.410956 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.410916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj8ck\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-kube-api-access-pj8ck\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.411125 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.410972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-bound-sa-token\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.511806 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.511770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj8ck\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-kube-api-access-pj8ck\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.511914 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.511826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-bound-sa-token\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.521435 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.521404 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-bound-sa-token\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.521650 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.521627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj8ck\" (UniqueName: \"kubernetes.io/projected/f275199c-7251-4f8d-bd69-387cd27f029d-kube-api-access-pj8ck\") pod \"cert-manager-55b4d954d5-hzp4g\" (UID: \"f275199c-7251-4f8d-bd69-387cd27f029d\") " pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.625941 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.625853 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hzp4g" Jul 07 15:49:20.761243 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:20.761207 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hzp4g"] Jul 07 15:49:20.764508 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:20.764479 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf275199c_7251_4f8d_bd69_387cd27f029d.slice/crio-4381200d845d9daff7ff61c67c25416c2a3ce030a74504a4abe70a46dfc02140 WatchSource:0}: Error finding container 4381200d845d9daff7ff61c67c25416c2a3ce030a74504a4abe70a46dfc02140: Status 404 returned error can't find the container with id 4381200d845d9daff7ff61c67c25416c2a3ce030a74504a4abe70a46dfc02140 Jul 07 15:49:21.356062 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:21.356025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hzp4g" event={"ID":"f275199c-7251-4f8d-bd69-387cd27f029d","Type":"ContainerStarted","Data":"45a98446bb09fb03274890b951f66efc4ccae77af5281160da4830cf99284f25"} Jul 07 15:49:21.356062 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:21.356069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hzp4g" event={"ID":"f275199c-7251-4f8d-bd69-387cd27f029d","Type":"ContainerStarted","Data":"4381200d845d9daff7ff61c67c25416c2a3ce030a74504a4abe70a46dfc02140"} Jul 07 15:49:21.376332 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:21.376277 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hzp4g" podStartSLOduration=1.376261066 podStartE2EDuration="1.376261066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:21.372744525 +0000 UTC m=+334.603810800" watchObservedRunningTime="2026-07-07 15:49:21.376261066 +0000 UTC m=+334.607327340" Jul 07 15:49:24.348036 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:24.348001 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kv4hx" Jul 07 15:49:25.257983 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:25.257951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:25.350236 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:25.350197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:27.049848 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:27.049812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:28.828590 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.828549 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7"] Jul 07 15:49:28.832030 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.832012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.834616 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.834593 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:49:28.834616 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.834608 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:49:28.835750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.835699 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ljfx4\"" Jul 07 15:49:28.840554 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.840534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7"] Jul 07 15:49:28.875416 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.875358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.875581 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.875449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpxw9\" (UniqueName: \"kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.875581 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.875487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.976709 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.976665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.976864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.976746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.976864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.976796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpxw9\" (UniqueName: \"kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.977145 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.977116 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.977269 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.977158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:28.986045 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:28.986020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpxw9\" (UniqueName: \"kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:29.142292 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:29.142167 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:29.274843 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:29.274804 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7"] Jul 07 15:49:29.278394 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:29.278353 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bcaaf2_cf1f_4232_a4e8_038e0bdb9256.slice/crio-960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e WatchSource:0}: Error finding container 960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e: Status 404 returned error can't find the container with id 960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e Jul 07 15:49:29.393075 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:29.392999 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerID="2648f5b7fcf0308ca5282d57a6f34c7c7f3caaa73e14d8c3c8ac0efa58882275" exitCode=0 Jul 07 15:49:29.393075 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:29.393042 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" event={"ID":"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256","Type":"ContainerDied","Data":"2648f5b7fcf0308ca5282d57a6f34c7c7f3caaa73e14d8c3c8ac0efa58882275"} Jul 07 15:49:29.393243 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:29.393095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" event={"ID":"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256","Type":"ContainerStarted","Data":"960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e"} Jul 07 15:49:30.399474 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:30.399443 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerID="7e27dc9aee2a56e8765331a08ab2013fdf214585968de6acd11d531a6b458204" exitCode=0 Jul 07 15:49:30.399861 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:30.399495 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" event={"ID":"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256","Type":"ContainerDied","Data":"7e27dc9aee2a56e8765331a08ab2013fdf214585968de6acd11d531a6b458204"} Jul 07 15:49:31.406481 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:31.406448 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerID="412f8e76af2489af1ec6dd0bde9316a2b2b6314c0c36126c5c7c983a33dd709c" exitCode=0 Jul 07 15:49:31.406877 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:31.406551 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" event={"ID":"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256","Type":"ContainerDied","Data":"412f8e76af2489af1ec6dd0bde9316a2b2b6314c0c36126c5c7c983a33dd709c"} Jul 07 15:49:32.537918 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.537892 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:32.605758 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.605725 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpxw9\" (UniqueName: \"kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9\") pod \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " Jul 07 15:49:32.605895 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.605809 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util\") pod \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " Jul 07 15:49:32.605895 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.605836 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle\") pod \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\" (UID: \"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256\") " Jul 07 15:49:32.606593 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.606564 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle" (OuterVolumeSpecName: "bundle") pod "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" (UID: "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:49:32.607846 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.607823 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9" (OuterVolumeSpecName: "kube-api-access-lpxw9") pod "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" (UID: "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256"). InnerVolumeSpecName "kube-api-access-lpxw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:49:32.611209 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.611186 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util" (OuterVolumeSpecName: "util") pod "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" (UID: "d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:49:32.706673 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.706570 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:32.706673 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.706615 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:32.706673 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:32.706625 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpxw9\" (UniqueName: \"kubernetes.io/projected/d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256-kube-api-access-lpxw9\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:49:33.417974 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:33.417945 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" Jul 07 15:49:33.418144 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:33.417973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8n6m7" event={"ID":"d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256","Type":"ContainerDied","Data":"960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e"} Jul 07 15:49:33.418144 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:33.418026 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="960fe02f6057b40b88e9536a47b052c63a26d995a5045c8885ce100e55184f6e" Jul 07 15:49:40.155982 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:40.155896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:40.259997 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:40.259964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:40.775956 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:40.775904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.101407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.101353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.205502 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.205459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.215885 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.215853 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx"] Jul 07 15:49:41.216179 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216166 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="pull" Jul 07 15:49:41.216217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216181 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="pull" Jul 07 15:49:41.216217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216191 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="util" Jul 07 15:49:41.216217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216197 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="util" Jul 07 15:49:41.216217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216206 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="extract" Jul 07 15:49:41.216217 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216213 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="extract" Jul 07 15:49:41.216361 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.216268 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2bcaaf2-cf1f-4232-a4e8-038e0bdb9256" containerName="extract" Jul 07 15:49:41.219365 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.219345 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.222453 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.222434 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:49:41.222839 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.222823 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hzz2w\"" Jul 07 15:49:41.223061 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.223037 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:49:41.223333 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.223313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:49:41.223451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.223364 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:49:41.235255 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.235228 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx"] Jul 07 15:49:41.278598 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.278564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbnb2\" (UniqueName: \"kubernetes.io/projected/da7e15c2-1fca-46f1-8d7a-137e57e813b6-kube-api-access-dbnb2\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.278598 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.278603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.278824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.278632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.278824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.278690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.278824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.278717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379189 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379189 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379189 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnb2\" (UniqueName: \"kubernetes.io/projected/da7e15c2-1fca-46f1-8d7a-137e57e813b6-kube-api-access-dbnb2\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379718 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379691 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-manifests\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.379995 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.379970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.381459 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.381437 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da7e15c2-1fca-46f1-8d7a-137e57e813b6-tmp\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.381736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.381715 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.381785 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.381729 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/da7e15c2-1fca-46f1-8d7a-137e57e813b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.393451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.393429 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbnb2\" (UniqueName: \"kubernetes.io/projected/da7e15c2-1fca-46f1-8d7a-137e57e813b6-kube-api-access-dbnb2\") pod \"opendatahub-operator-controller-manager-56f57b78d4-5sgtx\" (UID: \"da7e15c2-1fca-46f1-8d7a-137e57e813b6\") " pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.489119 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.489086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.530097 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.530062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:41.581739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.581707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.682064 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.682037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx"] Jul 07 15:49:41.684159 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:49:41.684128 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7e15c2_1fca_46f1_8d7a_137e57e813b6.slice/crio-69b8ec12bd2d7b6e568555ea46b4094d8232560f6a96429d77560b7fe215fcd9 WatchSource:0}: Error finding container 69b8ec12bd2d7b6e568555ea46b4094d8232560f6a96429d77560b7fe215fcd9: Status 404 returned error can't find the container with id 69b8ec12bd2d7b6e568555ea46b4094d8232560f6a96429d77560b7fe215fcd9 Jul 07 15:49:41.691274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.691244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:41.782285 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:41.782249 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:42.465254 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:42.465212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" event={"ID":"da7e15c2-1fca-46f1-8d7a-137e57e813b6","Type":"ContainerStarted","Data":"69b8ec12bd2d7b6e568555ea46b4094d8232560f6a96429d77560b7fe215fcd9"} Jul 07 15:49:43.950717 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:43.950679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:44.053860 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:44.053816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:45.480780 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:45.480743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" event={"ID":"da7e15c2-1fca-46f1-8d7a-137e57e813b6","Type":"ContainerStarted","Data":"6a86e4bec3eeb19e8bfcf8981818676d0a1c2a76813a00859219cd0bdddef00a"} Jul 07 15:49:46.486986 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:46.486950 2562 generic.go:356] "Generic (PLEG): container finished" podID="da7e15c2-1fca-46f1-8d7a-137e57e813b6" containerID="6a86e4bec3eeb19e8bfcf8981818676d0a1c2a76813a00859219cd0bdddef00a" exitCode=0 Jul 07 15:49:46.487427 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:46.486995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" event={"ID":"da7e15c2-1fca-46f1-8d7a-137e57e813b6","Type":"ContainerDied","Data":"6a86e4bec3eeb19e8bfcf8981818676d0a1c2a76813a00859219cd0bdddef00a"} Jul 07 15:49:47.493167 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:47.493133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" event={"ID":"da7e15c2-1fca-46f1-8d7a-137e57e813b6","Type":"ContainerStarted","Data":"d767d68464b4a95386fef0009cf0c738268124adba1723fb42c1041dda51dd83"} Jul 07 15:49:47.493601 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:47.493232 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:47.542415 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:47.542340 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" podStartSLOduration=2.113710896 podStartE2EDuration="6.542321768s" totalImagesPullingTime="3.365348903s" totalInitContainerRuntime="1.063261969s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:49:47.541661566 +0000 UTC m=+360.772727842" watchObservedRunningTime="2026-07-07 15:49:47.542321768 +0000 UTC m=+360.773388043" Jul 07 15:49:49.276170 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:49.276137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:49.378932 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:49.378897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:52.409744 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:52.409712 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:52.504268 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:52.504235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:49:58.500506 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:58.500473 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56f57b78d4-5sgtx" Jul 07 15:49:59.939330 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.939297 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt"] Jul 07 15:49:59.948004 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.947980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:49:59.950957 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.950933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 15:49:59.951203 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.951186 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 15:49:59.952242 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.952223 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:49:59.952579 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.952561 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 15:49:59.953816 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.953795 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vxprj\"" Jul 07 15:49:59.953907 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.953806 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:49:59.972146 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:49:59.972121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt"] Jul 07 15:50:00.037164 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.037128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb1dbe7a-56cf-4383-9320-09595a3f37bb-manager-config\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.037352 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.037183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.037352 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.037263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.037352 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.037313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29rpm\" (UniqueName: \"kubernetes.io/projected/eb1dbe7a-56cf-4383-9320-09595a3f37bb-kube-api-access-29rpm\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.138724 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.138684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.138909 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.138743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.138956 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.138922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29rpm\" (UniqueName: \"kubernetes.io/projected/eb1dbe7a-56cf-4383-9320-09595a3f37bb-kube-api-access-29rpm\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.139004 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.138980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb1dbe7a-56cf-4383-9320-09595a3f37bb-manager-config\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.139627 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.139607 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb1dbe7a-56cf-4383-9320-09595a3f37bb-manager-config\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.141296 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.141271 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-metrics-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.141440 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.141356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb1dbe7a-56cf-4383-9320-09595a3f37bb-cert\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.152984 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.152957 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29rpm\" (UniqueName: \"kubernetes.io/projected/eb1dbe7a-56cf-4383-9320-09595a3f37bb-kube-api-access-29rpm\") pod \"lws-controller-manager-646f7cc9db-cfnqt\" (UID: \"eb1dbe7a-56cf-4383-9320-09595a3f37bb\") " pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.257610 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.257531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:00.336079 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.336047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:00.412787 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:50:00.412758 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1dbe7a_56cf_4383_9320_09595a3f37bb.slice/crio-616f1bed115acf3ae8aba20237e770d515e0acc2b710deb61c976f2d0dbab1b1 WatchSource:0}: Error finding container 616f1bed115acf3ae8aba20237e770d515e0acc2b710deb61c976f2d0dbab1b1: Status 404 returned error can't find the container with id 616f1bed115acf3ae8aba20237e770d515e0acc2b710deb61c976f2d0dbab1b1 Jul 07 15:50:00.420490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.420459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt"] Jul 07 15:50:00.468872 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.468836 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:00.551983 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.551947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" event={"ID":"eb1dbe7a-56cf-4383-9320-09595a3f37bb","Type":"ContainerStarted","Data":"616f1bed115acf3ae8aba20237e770d515e0acc2b710deb61c976f2d0dbab1b1"} Jul 07 15:50:00.736603 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.736559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:00.935660 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:00.935557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:01.270003 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:01.269849 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:01.501692 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:01.501653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:01.680232 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:01.680210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:02.008658 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.008543 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:02.344124 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.344069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:02.562558 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.562513 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" event={"ID":"eb1dbe7a-56cf-4383-9320-09595a3f37bb","Type":"ContainerStarted","Data":"e2094e36ab03e250c545d6d027e398e9f09844f41fa54809e0b50c3d2d7fea7d"} Jul 07 15:50:02.562795 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.562591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:02.584551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.584489 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" podStartSLOduration=2.376012771 podStartE2EDuration="3.584466872s" totalImagesPullingTime="1.208454101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:02.583477207 +0000 UTC m=+375.814543496" watchObservedRunningTime="2026-07-07 15:50:02.584466872 +0000 UTC m=+375.815533143" Jul 07 15:50:02.960501 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:02.960467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:03.045626 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:03.045560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:04.483161 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:04.483112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:08.192447 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.192405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:08.291718 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.291687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:08.843529 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.843498 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp"] Jul 07 15:50:08.847941 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.847922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:08.850921 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.850897 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 15:50:08.851027 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.850901 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-x9nl9\"" Jul 07 15:50:08.863121 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.863097 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp"] Jul 07 15:50:08.913143 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:08.913105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:09.016777 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.016777 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017025 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017025 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017025 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2k78\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-kube-api-access-n2k78\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017025 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017025 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.016952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.017037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.017274 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.017068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.118300 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.118201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.118595 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.118574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.119110 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.118688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.119255 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.119239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120108 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2k78\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-kube-api-access-n2k78\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120394 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120470 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120027 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120470 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120804 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120927 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120841 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.120927 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.120910 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.122786 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.122758 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.123050 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.123029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.138220 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.138199 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2k78\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-kube-api-access-n2k78\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.138459 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.138442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce19fbd-5c0f-46fa-9284-64147fd126fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678x64lp\" (UID: \"6ce19fbd-5c0f-46fa-9284-64147fd126fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.159558 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.159533 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:09.240333 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.240302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:09.327462 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.326795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp"] Jul 07 15:50:09.594941 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.594900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" event={"ID":"6ce19fbd-5c0f-46fa-9284-64147fd126fd","Type":"ContainerStarted","Data":"b4ce4f31813e1435e27dfe78aa3575256a77b645a4f9354cc03c5eb21f7fffa4"} Jul 07 15:50:09.898125 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:09.898039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:11.365681 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.365647 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c"] Jul 07 15:50:11.372638 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.372616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.375678 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.375650 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 15:50:11.375820 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.375683 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-txbh7\"" Jul 07 15:50:11.375943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.375925 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 15:50:11.376014 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.375926 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 15:50:11.396048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.396014 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c"] Jul 07 15:50:11.542759 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.542951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.542951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e00134-3b20-4048-afcb-b55bce2690a0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.542951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.542951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.542951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.542929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.543159 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.543021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtp4h\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-kube-api-access-wtp4h\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtp4h\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-kube-api-access-wtp4h\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644594 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e00134-3b20-4048-afcb-b55bce2690a0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644594 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644594 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.644594 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.644584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.645290 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.645259 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.647192 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.647135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e00134-3b20-4048-afcb-b55bce2690a0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.647330 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.647306 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.647453 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.647425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.647579 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.647562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56e00134-3b20-4048-afcb-b55bce2690a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.654659 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.654634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.654939 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.654921 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtp4h\" (UniqueName: \"kubernetes.io/projected/56e00134-3b20-4048-afcb-b55bce2690a0-kube-api-access-wtp4h\") pod \"istiod-openshift-gateway-94bb8fbfd-gj42c\" (UID: \"56e00134-3b20-4048-afcb-b55bce2690a0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:11.683999 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:11.683962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:12.139867 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.139815 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:50:12.139983 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.139921 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:50:12.139983 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.139967 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:50:12.237885 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.237859 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c"] Jul 07 15:50:12.239881 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:50:12.239845 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e00134_3b20_4048_afcb_b55bce2690a0.slice/crio-c28eb91e936eeea09c31b65adc0814bae564d672c50ab7f3795aee5d4e65cd82 WatchSource:0}: Error finding container c28eb91e936eeea09c31b65adc0814bae564d672c50ab7f3795aee5d4e65cd82: Status 404 returned error can't find the container with id c28eb91e936eeea09c31b65adc0814bae564d672c50ab7f3795aee5d4e65cd82 Jul 07 15:50:12.614778 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.614741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" event={"ID":"56e00134-3b20-4048-afcb-b55bce2690a0","Type":"ContainerStarted","Data":"c28eb91e936eeea09c31b65adc0814bae564d672c50ab7f3795aee5d4e65cd82"} Jul 07 15:50:12.616353 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.616314 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" event={"ID":"6ce19fbd-5c0f-46fa-9284-64147fd126fd","Type":"ContainerStarted","Data":"7276bb5582e08794db7beb5868398fe17219c865d21f4edc8e7cbe82f78d0f38"} Jul 07 15:50:12.682451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.682400 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" podStartSLOduration=1.874555568 podStartE2EDuration="4.682355849s" totalImagesPullingTime="2.807800281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:12.674196021 +0000 UTC m=+385.905262299" watchObservedRunningTime="2026-07-07 15:50:12.682355849 +0000 UTC m=+385.913422123" Jul 07 15:50:12.747085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.747023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:12.884553 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.884466 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:12.996450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:12.996415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:13.098855 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.098820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:13.160585 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.160485 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:13.167123 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.167089 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:13.207744 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.207693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:13.306168 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.306126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:13.422951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.422864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:13.570583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.570554 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-646f7cc9db-cfnqt" Jul 07 15:50:13.622588 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.622423 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:13.624042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:13.624020 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678x64lp" Jul 07 15:50:14.493921 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:14.493869 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:50:14.494040 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:14.493973 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:50:14.628046 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:14.628011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" event={"ID":"56e00134-3b20-4048-afcb-b55bce2690a0","Type":"ContainerStarted","Data":"4679d14c4fa1f3357798feafb3d57a6b922ce709e890f398e2514fc748d4c1a3"} Jul 07 15:50:14.628551 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:14.628097 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:14.657250 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:14.657185 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" podStartSLOduration=1.405652607 podStartE2EDuration="3.65716732s" totalImagesPullingTime="2.251514713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:14.654702773 +0000 UTC m=+387.885769099" watchObservedRunningTime="2026-07-07 15:50:14.65716732 +0000 UTC m=+387.888233594" Jul 07 15:50:15.635223 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:15.635194 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gj42c" Jul 07 15:50:43.829911 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:43.829879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:45.448450 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.448418 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b9d9c5675-k9k9z"] Jul 07 15:50:45.454833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.454808 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.467959 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.467932 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9d9c5675-k9k9z"] Jul 07 15:50:45.526642 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.526608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:45.550622 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtkfq\" (UniqueName: \"kubernetes.io/projected/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-kube-api-access-jtkfq\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.550823 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.550823 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-oauth-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.550943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-service-ca\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.550943 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.551042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-trusted-ca-bundle\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.551042 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.550994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-oauth-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.622903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.622866 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:45.651864 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-oauth-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-service-ca\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-trusted-ca-bundle\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-oauth-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtkfq\" (UniqueName: \"kubernetes.io/projected/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-kube-api-access-jtkfq\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652067 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.651989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652830 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.652802 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-service-ca\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652830 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.652818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.652830 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.652824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-oauth-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.653022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.652896 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-trusted-ca-bundle\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.654729 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.654697 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-serving-cert\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.654729 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.654713 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-console-oauth-config\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.661225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.661203 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtkfq\" (UniqueName: \"kubernetes.io/projected/e00a76f6-fb8c-4be9-9d4f-5691b7e81f96-kube-api-access-jtkfq\") pod \"console-6b9d9c5675-k9k9z\" (UID: \"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96\") " pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.764719 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.764633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:45.781644 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.781605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:45.910501 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:45.910476 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9d9c5675-k9k9z"] Jul 07 15:50:45.913721 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:50:45.913680 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00a76f6_fb8c_4be9_9d4f_5691b7e81f96.slice/crio-fa2c8d79d49b75178a7e2afba248fd0127d94fbd6ee725bf175fc78abfdfbebe WatchSource:0}: Error finding container fa2c8d79d49b75178a7e2afba248fd0127d94fbd6ee725bf175fc78abfdfbebe: Status 404 returned error can't find the container with id fa2c8d79d49b75178a7e2afba248fd0127d94fbd6ee725bf175fc78abfdfbebe Jul 07 15:50:46.789822 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:46.789765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d9c5675-k9k9z" event={"ID":"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96","Type":"ContainerStarted","Data":"f7ded9bde815ca9c3153e261284322b32bd6c3ace05247daf84f34b15c533c6e"} Jul 07 15:50:46.790236 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:46.789868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d9c5675-k9k9z" event={"ID":"e00a76f6-fb8c-4be9-9d4f-5691b7e81f96","Type":"ContainerStarted","Data":"fa2c8d79d49b75178a7e2afba248fd0127d94fbd6ee725bf175fc78abfdfbebe"} Jul 07 15:50:46.816422 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:46.816354 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b9d9c5675-k9k9z" podStartSLOduration=1.8163390750000001 podStartE2EDuration="1.816339075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:50:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:50:46.814784055 +0000 UTC m=+420.045850363" watchObservedRunningTime="2026-07-07 15:50:46.816339075 +0000 UTC m=+420.047405350" Jul 07 15:50:55.765736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:55.765686 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:55.766167 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:55.765758 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:55.770515 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:55.770489 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:55.842929 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:55.842901 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b9d9c5675-k9k9z" Jul 07 15:50:55.894583 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:55.894539 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:50:56.052002 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:56.051968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:50:56.207736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:50:56.207705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:01.728587 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:01.728552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:01.831451 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:01.831419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:01.901732 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:01.901702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:02.664834 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:02.664807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:02.876572 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:02.876541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:18.078170 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.078074 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc"] Jul 07 15:51:18.081880 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.081857 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.084730 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.084702 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:51:18.084847 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.084761 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:51:18.085825 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.085806 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hmlvg\"" Jul 07 15:51:18.091625 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.091603 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc"] Jul 07 15:51:18.139747 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.139688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvczk\" (UniqueName: \"kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.139747 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.139758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.139981 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.139807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.240642 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.240586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvczk\" (UniqueName: \"kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.240857 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.240662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.240857 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.240702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.241060 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.241036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.241128 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.241110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.249847 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.249817 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvczk\" (UniqueName: \"kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.392206 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.392101 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:18.474145 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.474110 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5"] Jul 07 15:51:18.478886 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.478570 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.490987 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.490953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5"] Jul 07 15:51:18.535430 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.535406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc"] Jul 07 15:51:18.537153 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:51:18.537129 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777ffd8a_0ab9_47ea_b2a3_000f817122f9.slice/crio-ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0 WatchSource:0}: Error finding container ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0: Status 404 returned error can't find the container with id ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0 Jul 07 15:51:18.542905 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.542875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pqpj\" (UniqueName: \"kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.543029 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.542936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.543029 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.542993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.643779 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.643711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.643779 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.643760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.644020 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.643822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pqpj\" (UniqueName: \"kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.644159 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.644135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.644199 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.644143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.652886 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.652858 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pqpj\" (UniqueName: \"kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.790491 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.790451 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:18.863785 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.863755 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586"] Jul 07 15:51:18.868326 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.868306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:18.882628 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.882595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586"] Jul 07 15:51:18.927762 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.927736 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5"] Jul 07 15:51:18.929303 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:51:18.929275 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c92dc76_e518_4e68_930e_97de2c0f8c50.slice/crio-1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8 WatchSource:0}: Error finding container 1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8: Status 404 returned error can't find the container with id 1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8 Jul 07 15:51:18.946933 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.946905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:18.947045 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.946966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487b5\" (UniqueName: \"kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:18.947108 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.947032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:18.951536 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.951514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" event={"ID":"7c92dc76-e518-4e68-930e-97de2c0f8c50","Type":"ContainerStarted","Data":"1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8"} Jul 07 15:51:18.952803 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.952783 2562 generic.go:356] "Generic (PLEG): container finished" podID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerID="779496d6f64ac9dea1f5970fcc114334aa7799d34abf019ac24399362b13e17a" exitCode=0 Jul 07 15:51:18.952881 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.952861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" event={"ID":"777ffd8a-0ab9-47ea-b2a3-000f817122f9","Type":"ContainerDied","Data":"779496d6f64ac9dea1f5970fcc114334aa7799d34abf019ac24399362b13e17a"} Jul 07 15:51:18.952928 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:18.952902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" event={"ID":"777ffd8a-0ab9-47ea-b2a3-000f817122f9","Type":"ContainerStarted","Data":"ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0"} Jul 07 15:51:19.048523 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.048490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.048676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.048537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487b5\" (UniqueName: \"kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.048676 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.048558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.048932 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.048910 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.048932 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.048927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.057053 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.057031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-487b5\" (UniqueName: \"kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.179456 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.179350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:19.268024 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.267992 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7"] Jul 07 15:51:19.271901 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.271881 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.282250 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.282226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7"] Jul 07 15:51:19.339104 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.339076 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586"] Jul 07 15:51:19.340927 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:51:19.340890 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3939077_bd1f_448f_a0f2_8e230c76872c.slice/crio-41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4 WatchSource:0}: Error finding container 41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4: Status 404 returned error can't find the container with id 41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4 Jul 07 15:51:19.350851 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.350828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvzfd\" (UniqueName: \"kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.350938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.350880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.350938 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.350897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.451790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.451700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvzfd\" (UniqueName: \"kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.451790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.451753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.451790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.451771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.452207 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.452179 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.452207 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.452201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.461444 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.461409 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvzfd\" (UniqueName: \"kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.583114 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.583074 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:19.727883 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.727858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7"] Jul 07 15:51:19.730741 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:51:19.730707 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a00f97_673d_434f_9593_e609aed806ec.slice/crio-34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370 WatchSource:0}: Error finding container 34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370: Status 404 returned error can't find the container with id 34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370 Jul 07 15:51:19.959272 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.959233 2562 generic.go:356] "Generic (PLEG): container finished" podID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerID="56455926fc8e884adbfe3f5429296070dcd8f07e920b56be1332b209f230855d" exitCode=0 Jul 07 15:51:19.959483 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.959325 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" event={"ID":"7c92dc76-e518-4e68-930e-97de2c0f8c50","Type":"ContainerDied","Data":"56455926fc8e884adbfe3f5429296070dcd8f07e920b56be1332b209f230855d"} Jul 07 15:51:19.961156 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.961129 2562 generic.go:356] "Generic (PLEG): container finished" podID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerID="97beb88c3f3966c334d24deab995a6be394cfa95063c0676e638bd74eeaddda0" exitCode=0 Jul 07 15:51:19.961271 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.961224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" event={"ID":"777ffd8a-0ab9-47ea-b2a3-000f817122f9","Type":"ContainerDied","Data":"97beb88c3f3966c334d24deab995a6be394cfa95063c0676e638bd74eeaddda0"} Jul 07 15:51:19.962942 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.962917 2562 generic.go:356] "Generic (PLEG): container finished" podID="67a00f97-673d-434f-9593-e609aed806ec" containerID="676282f1af1210a359bea6376ec598af25ce551e0412af3e924ede8985be5e14" exitCode=0 Jul 07 15:51:19.963017 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.962986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" event={"ID":"67a00f97-673d-434f-9593-e609aed806ec","Type":"ContainerDied","Data":"676282f1af1210a359bea6376ec598af25ce551e0412af3e924ede8985be5e14"} Jul 07 15:51:19.963076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.963029 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" event={"ID":"67a00f97-673d-434f-9593-e609aed806ec","Type":"ContainerStarted","Data":"34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370"} Jul 07 15:51:19.964724 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.964702 2562 generic.go:356] "Generic (PLEG): container finished" podID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerID="7692fb466ca19166e381ccd7e3ac99d55023bf4b59b79169db61bf18541f64c6" exitCode=0 Jul 07 15:51:19.964833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.964735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerDied","Data":"7692fb466ca19166e381ccd7e3ac99d55023bf4b59b79169db61bf18541f64c6"} Jul 07 15:51:19.964833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:19.964767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerStarted","Data":"41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4"} Jul 07 15:51:20.922810 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.922776 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6859fbc6c9-d6bt7" podUID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" containerName="console" containerID="cri-o://ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6" gracePeriod=15 Jul 07 15:51:20.972060 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.972019 2562 generic.go:356] "Generic (PLEG): container finished" podID="67a00f97-673d-434f-9593-e609aed806ec" containerID="a6dc9af51d19229d2e1fbea1aa9a3506da98e4965cdc31858283fc841b564d2d" exitCode=0 Jul 07 15:51:20.972222 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.972105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" event={"ID":"67a00f97-673d-434f-9593-e609aed806ec","Type":"ContainerDied","Data":"a6dc9af51d19229d2e1fbea1aa9a3506da98e4965cdc31858283fc841b564d2d"} Jul 07 15:51:20.974001 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.973971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerStarted","Data":"c3feea25a49008adbdd4cfdf7f4d53d5b68f6d578c4530e1577cefc02c258670"} Jul 07 15:51:20.976040 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.976014 2562 generic.go:356] "Generic (PLEG): container finished" podID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerID="bcf7de9cc849b3d0fc7e3fb6145b31d4e9992286c09b95426ad4e398f0f5f542" exitCode=0 Jul 07 15:51:20.976137 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.976067 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" event={"ID":"7c92dc76-e518-4e68-930e-97de2c0f8c50","Type":"ContainerDied","Data":"bcf7de9cc849b3d0fc7e3fb6145b31d4e9992286c09b95426ad4e398f0f5f542"} Jul 07 15:51:20.978426 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.978406 2562 generic.go:356] "Generic (PLEG): container finished" podID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerID="fb32e9739a6affb3821b272cc3ba08549ec6d7965892defc1fb37c829114c4ee" exitCode=0 Jul 07 15:51:20.978513 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:20.978471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" event={"ID":"777ffd8a-0ab9-47ea-b2a3-000f817122f9","Type":"ContainerDied","Data":"fb32e9739a6affb3821b272cc3ba08549ec6d7965892defc1fb37c829114c4ee"} Jul 07 15:51:21.163888 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.163853 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:51:21.267455 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267428 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267567 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267487 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x726h\" (UniqueName: \"kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267567 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267516 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267567 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267552 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267865 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267569 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267865 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267634 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267865 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert\") pod \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\" (UID: \"205c3cea-c3cf-4273-bd06-4b8f5233f52e\") " Jul 07 15:51:21.267865 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267840 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca" (OuterVolumeSpecName: "service-ca") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:51:21.268076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267955 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-service-ca\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.268076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.267993 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:51:21.268076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.268044 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:51:21.268371 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.268348 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config" (OuterVolumeSpecName: "console-config") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:51:21.269876 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.269853 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:51:21.270085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.270070 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h" (OuterVolumeSpecName: "kube-api-access-x726h") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "kube-api-access-x726h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:51:21.270133 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.270103 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "205c3cea-c3cf-4273-bd06-4b8f5233f52e" (UID: "205c3cea-c3cf-4273-bd06-4b8f5233f52e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:51:21.368444 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368404 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.368614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368456 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x726h\" (UniqueName: \"kubernetes.io/projected/205c3cea-c3cf-4273-bd06-4b8f5233f52e-kube-api-access-x726h\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.368614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368467 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-oauth-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.368614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368476 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-console-config\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.368614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368485 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-oauth-serving-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.368614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.368494 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/205c3cea-c3cf-4273-bd06-4b8f5233f52e-trusted-ca-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:21.986058 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.986024 2562 generic.go:356] "Generic (PLEG): container finished" podID="67a00f97-673d-434f-9593-e609aed806ec" containerID="4f0bb2cc093571848db1eae90e35fd38c57e971c9f97df5762157fe8d9f4ade5" exitCode=0 Jul 07 15:51:21.986471 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.986096 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" event={"ID":"67a00f97-673d-434f-9593-e609aed806ec","Type":"ContainerDied","Data":"4f0bb2cc093571848db1eae90e35fd38c57e971c9f97df5762157fe8d9f4ade5"} Jul 07 15:51:21.987823 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.987794 2562 generic.go:356] "Generic (PLEG): container finished" podID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerID="c3feea25a49008adbdd4cfdf7f4d53d5b68f6d578c4530e1577cefc02c258670" exitCode=0 Jul 07 15:51:21.987939 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.987878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerDied","Data":"c3feea25a49008adbdd4cfdf7f4d53d5b68f6d578c4530e1577cefc02c258670"} Jul 07 15:51:21.990105 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.990083 2562 generic.go:356] "Generic (PLEG): container finished" podID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerID="6d01b4729370f2fdb1805445b97ee5776b9946c1d88a02db440b493058a06fb2" exitCode=0 Jul 07 15:51:21.990249 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.990172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" event={"ID":"7c92dc76-e518-4e68-930e-97de2c0f8c50","Type":"ContainerDied","Data":"6d01b4729370f2fdb1805445b97ee5776b9946c1d88a02db440b493058a06fb2"} Jul 07 15:51:21.991447 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.991425 2562 generic.go:356] "Generic (PLEG): container finished" podID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" containerID="ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6" exitCode=0 Jul 07 15:51:21.991609 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.991483 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859fbc6c9-d6bt7" Jul 07 15:51:21.991609 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.991502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859fbc6c9-d6bt7" event={"ID":"205c3cea-c3cf-4273-bd06-4b8f5233f52e","Type":"ContainerDied","Data":"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6"} Jul 07 15:51:21.991609 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.991542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859fbc6c9-d6bt7" event={"ID":"205c3cea-c3cf-4273-bd06-4b8f5233f52e","Type":"ContainerDied","Data":"b2d634e3f0a6fb9023243c4d49a96d1b30c2fc82ea37fff62ddddc8fafc0b18d"} Jul 07 15:51:21.991609 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:21.991562 2562 scope.go:117] "RemoveContainer" containerID="ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6" Jul 07 15:51:22.003767 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.003745 2562 scope.go:117] "RemoveContainer" containerID="ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6" Jul 07 15:51:22.004098 ip-10-0-141-42 kubenswrapper[2562]: E0707 15:51:22.004076 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6\": container with ID starting with ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6 not found: ID does not exist" containerID="ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6" Jul 07 15:51:22.004164 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.004106 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6"} err="failed to get container status \"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6\": rpc error: code = NotFound desc = could not find container \"ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6\": container with ID starting with ef98a80a48f29b16e6c6395ed59f72a459a28f1170ead7ea9db8f746d6a230f6 not found: ID does not exist" Jul 07 15:51:22.062405 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.061732 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:51:22.064604 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.064578 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6859fbc6c9-d6bt7"] Jul 07 15:51:22.132614 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.132581 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:22.175370 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.175340 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle\") pod \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " Jul 07 15:51:22.175564 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.175434 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util\") pod \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " Jul 07 15:51:22.175564 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.175493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvczk\" (UniqueName: \"kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk\") pod \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\" (UID: \"777ffd8a-0ab9-47ea-b2a3-000f817122f9\") " Jul 07 15:51:22.175968 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.175935 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle" (OuterVolumeSpecName: "bundle") pod "777ffd8a-0ab9-47ea-b2a3-000f817122f9" (UID: "777ffd8a-0ab9-47ea-b2a3-000f817122f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:22.177824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.177791 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk" (OuterVolumeSpecName: "kube-api-access-bvczk") pod "777ffd8a-0ab9-47ea-b2a3-000f817122f9" (UID: "777ffd8a-0ab9-47ea-b2a3-000f817122f9"). InnerVolumeSpecName "kube-api-access-bvczk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:51:22.180273 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.180242 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util" (OuterVolumeSpecName: "util") pod "777ffd8a-0ab9-47ea-b2a3-000f817122f9" (UID: "777ffd8a-0ab9-47ea-b2a3-000f817122f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:22.277005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.276972 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvczk\" (UniqueName: \"kubernetes.io/projected/777ffd8a-0ab9-47ea-b2a3-000f817122f9-kube-api-access-bvczk\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:22.277005 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.277006 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:22.277152 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.277015 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/777ffd8a-0ab9-47ea-b2a3-000f817122f9-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:22.998986 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.998946 2562 generic.go:356] "Generic (PLEG): container finished" podID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerID="364e8ef9b713e1e35306ff9ae4aa9eaf56b8affda8ee1b3838c790045bf24c56" exitCode=0 Jul 07 15:51:22.999438 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:22.999037 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerDied","Data":"364e8ef9b713e1e35306ff9ae4aa9eaf56b8affda8ee1b3838c790045bf24c56"} Jul 07 15:51:23.000829 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.000810 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" Jul 07 15:51:23.000914 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.000845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc" event={"ID":"777ffd8a-0ab9-47ea-b2a3-000f817122f9","Type":"ContainerDied","Data":"ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0"} Jul 07 15:51:23.000914 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.000878 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac32b0f1bd42d9fa1446ab2011c3c7d547f167ca5efbedcdede716a6284393a0" Jul 07 15:51:23.171054 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.171028 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:23.173129 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.173110 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:23.285696 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285597 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util\") pod \"7c92dc76-e518-4e68-930e-97de2c0f8c50\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " Jul 07 15:51:23.285696 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285661 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util\") pod \"67a00f97-673d-434f-9593-e609aed806ec\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " Jul 07 15:51:23.285696 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285688 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvzfd\" (UniqueName: \"kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd\") pod \"67a00f97-673d-434f-9593-e609aed806ec\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " Jul 07 15:51:23.286048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285720 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle\") pod \"67a00f97-673d-434f-9593-e609aed806ec\" (UID: \"67a00f97-673d-434f-9593-e609aed806ec\") " Jul 07 15:51:23.286048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285748 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle\") pod \"7c92dc76-e518-4e68-930e-97de2c0f8c50\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " Jul 07 15:51:23.286048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.285780 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pqpj\" (UniqueName: \"kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj\") pod \"7c92dc76-e518-4e68-930e-97de2c0f8c50\" (UID: \"7c92dc76-e518-4e68-930e-97de2c0f8c50\") " Jul 07 15:51:23.286264 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.286233 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle" (OuterVolumeSpecName: "bundle") pod "7c92dc76-e518-4e68-930e-97de2c0f8c50" (UID: "7c92dc76-e518-4e68-930e-97de2c0f8c50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:23.286410 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.286326 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle" (OuterVolumeSpecName: "bundle") pod "67a00f97-673d-434f-9593-e609aed806ec" (UID: "67a00f97-673d-434f-9593-e609aed806ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:23.287953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.287927 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd" (OuterVolumeSpecName: "kube-api-access-vvzfd") pod "67a00f97-673d-434f-9593-e609aed806ec" (UID: "67a00f97-673d-434f-9593-e609aed806ec"). InnerVolumeSpecName "kube-api-access-vvzfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:51:23.288122 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.288097 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj" (OuterVolumeSpecName: "kube-api-access-7pqpj") pod "7c92dc76-e518-4e68-930e-97de2c0f8c50" (UID: "7c92dc76-e518-4e68-930e-97de2c0f8c50"). InnerVolumeSpecName "kube-api-access-7pqpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:51:23.292448 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.292416 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util" (OuterVolumeSpecName: "util") pod "7c92dc76-e518-4e68-930e-97de2c0f8c50" (UID: "7c92dc76-e518-4e68-930e-97de2c0f8c50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:23.293861 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.293837 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util" (OuterVolumeSpecName: "util") pod "67a00f97-673d-434f-9593-e609aed806ec" (UID: "67a00f97-673d-434f-9593-e609aed806ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:23.387085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387048 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pqpj\" (UniqueName: \"kubernetes.io/projected/7c92dc76-e518-4e68-930e-97de2c0f8c50-kube-api-access-7pqpj\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.387085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387080 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.387085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387089 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.387294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387100 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvzfd\" (UniqueName: \"kubernetes.io/projected/67a00f97-673d-434f-9593-e609aed806ec-kube-api-access-vvzfd\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.387294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387110 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a00f97-673d-434f-9593-e609aed806ec-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.387294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.387119 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c92dc76-e518-4e68-930e-97de2c0f8c50-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:23.925955 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:23.925913 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" path="/var/lib/kubelet/pods/205c3cea-c3cf-4273-bd06-4b8f5233f52e/volumes" Jul 07 15:51:24.007621 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.007594 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" Jul 07 15:51:24.008012 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.007593 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7" event={"ID":"67a00f97-673d-434f-9593-e609aed806ec","Type":"ContainerDied","Data":"34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370"} Jul 07 15:51:24.008012 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.007710 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34e5cb007ac41036188ca3df284f0580f2ba427a7b9a489f48bf43b45eafe370" Jul 07 15:51:24.009576 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.009548 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" Jul 07 15:51:24.009693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.009600 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5" event={"ID":"7c92dc76-e518-4e68-930e-97de2c0f8c50","Type":"ContainerDied","Data":"1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8"} Jul 07 15:51:24.009693 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.009642 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c16ea6307623865efa4c32788bd76554b3601cc16e326c7d5ec6dec27dca9e8" Jul 07 15:51:24.139872 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.139845 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:24.192350 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.192257 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-487b5\" (UniqueName: \"kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5\") pod \"c3939077-bd1f-448f-a0f2-8e230c76872c\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " Jul 07 15:51:24.192350 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.192300 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle\") pod \"c3939077-bd1f-448f-a0f2-8e230c76872c\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " Jul 07 15:51:24.192608 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.192356 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util\") pod \"c3939077-bd1f-448f-a0f2-8e230c76872c\" (UID: \"c3939077-bd1f-448f-a0f2-8e230c76872c\") " Jul 07 15:51:24.192894 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.192867 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle" (OuterVolumeSpecName: "bundle") pod "c3939077-bd1f-448f-a0f2-8e230c76872c" (UID: "c3939077-bd1f-448f-a0f2-8e230c76872c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:24.194362 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.194332 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5" (OuterVolumeSpecName: "kube-api-access-487b5") pod "c3939077-bd1f-448f-a0f2-8e230c76872c" (UID: "c3939077-bd1f-448f-a0f2-8e230c76872c"). InnerVolumeSpecName "kube-api-access-487b5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:51:24.199836 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.199808 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util" (OuterVolumeSpecName: "util") pod "c3939077-bd1f-448f-a0f2-8e230c76872c" (UID: "c3939077-bd1f-448f-a0f2-8e230c76872c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:51:24.293076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.293036 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-487b5\" (UniqueName: \"kubernetes.io/projected/c3939077-bd1f-448f-a0f2-8e230c76872c-kube-api-access-487b5\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:24.293076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.293070 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-bundle\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:24.293076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:24.293079 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3939077-bd1f-448f-a0f2-8e230c76872c-util\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 15:51:25.016040 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:25.015998 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" event={"ID":"c3939077-bd1f-448f-a0f2-8e230c76872c","Type":"ContainerDied","Data":"41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4"} Jul 07 15:51:25.016040 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:25.016034 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586" Jul 07 15:51:25.016491 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:25.016052 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41c54d2ff2c634603874cae13f83f79e675efdbd7086828a4b0e060f112e26a4" Jul 07 15:51:31.151528 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:31.151496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:35.354444 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:35.354402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:38.170294 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:38.170257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:41.565490 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:41.565448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:42.727925 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:42.727884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:42.981076 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:42.980988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:43.332304 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:43.332273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:44.793581 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:44.793548 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:47.385660 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:47.385627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:48.342613 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:48.342573 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:48.437547 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:48.437507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:50.248085 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:50.248050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:50.456326 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:50.456275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:52.155991 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:52.155954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:52.255424 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:52.255394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:55.731584 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:55.731541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:51:55.833215 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:51:55.833184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:03.600759 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:03.600716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:03.701578 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:03.701538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:06.092531 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:06.092492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:06.193251 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:06.193214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:08.929225 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:08.929192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:09.038493 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:09.038458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:12.473121 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:12.473063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:12.579634 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:12.579587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:14.113264 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:14.113214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:21.308577 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:21.308521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:21.405736 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:21.405688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:23.913771 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:23.913725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:24.015619 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:24.015586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:27.105624 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:27.105550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:27.206017 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:27.205985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:28.969446 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:28.969393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:29.128136 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:29.128099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:30.705792 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:30.705741 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:30.806850 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:30.806803 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:34.778035 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:34.778004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:34.846670 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:34.846638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:40.510439 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:40.510335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:40.608798 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:40.608760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:40.715547 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:40.715517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:40.780044 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:40.779927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:44.527940 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:44.527897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:44.645815 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:44.645774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:44.709595 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:44.709563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:45.461208 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:45.461164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:52:45.562879 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:52:45.562848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:01.421750 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:01.421714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:01.523068 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:01.523036 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:04.968267 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:04.968232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:05.069148 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:05.069108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:05.459576 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:05.459547 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:17.945946 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:17.945899 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:18.188258 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:18.188227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:24.056228 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:24.056190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:24.151673 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:24.151641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:24.447226 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:24.447143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:28.195441 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:28.195405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:28.284976 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:28.284942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:53:47.341519 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:47.341480 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:53:47.344134 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:47.344110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:53:47.349093 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:47.349070 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:53:47.351850 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:53:47.351832 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:54:00.599148 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:00.599113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:01.010162 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.010078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:01.234299 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.234267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:01.516895 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.516869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:01.599977 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.599942 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r9vh4"] Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600299 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="util" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600311 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="util" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600319 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600324 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600333 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" containerName="console" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600339 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" containerName="console" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600351 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600356 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600361 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600366 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600372 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600389 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="pull" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600395 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="extract" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600403 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="extract" Jul 07 15:54:01.600407 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600411 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600417 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600433 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600438 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600446 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600451 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600460 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600465 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600470 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600475 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600485 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600490 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="util" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600542 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="205c3cea-c3cf-4273-bd06-4b8f5233f52e" containerName="console" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600549 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="777ffd8a-0ab9-47ea-b2a3-000f817122f9" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600555 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c92dc76-e518-4e68-930e-97de2c0f8c50" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600561 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a00f97-673d-434f-9593-e609aed806ec" containerName="extract" Jul 07 15:54:01.600853 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.600566 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3939077-bd1f-448f-a0f2-8e230c76872c" containerName="extract" Jul 07 15:54:01.603603 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.603585 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.606422 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.606398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-p8bnj\"" Jul 07 15:54:01.606562 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.606400 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 15:54:01.613666 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.613638 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r9vh4"] Jul 07 15:54:01.625903 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.625865 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mv6k"] Jul 07 15:54:01.629263 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.629239 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.646963 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.646933 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mv6k"] Jul 07 15:54:01.669003 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.668971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.669003 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.669015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19930a0b-e995-434a-8bf1-d1040522618f-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.669237 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.669095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzqd6\" (UniqueName: \"kubernetes.io/projected/19930a0b-e995-434a-8bf1-d1040522618f-kube-api-access-lzqd6\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.669237 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.669161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ldkn\" (UniqueName: \"kubernetes.io/projected/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-kube-api-access-9ldkn\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.670211 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.670186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:01.769896 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.769797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ldkn\" (UniqueName: \"kubernetes.io/projected/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-kube-api-access-9ldkn\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.769896 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.769860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.769896 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.769885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19930a0b-e995-434a-8bf1-d1040522618f-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.770236 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.769911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzqd6\" (UniqueName: \"kubernetes.io/projected/19930a0b-e995-434a-8bf1-d1040522618f-kube-api-access-lzqd6\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.770685 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.770667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19930a0b-e995-434a-8bf1-d1040522618f-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.770739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.770724 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.779560 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.779520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzqd6\" (UniqueName: \"kubernetes.io/projected/19930a0b-e995-434a-8bf1-d1040522618f-kube-api-access-lzqd6\") pod \"payload-processing-58f86c7798-5mv6k\" (UID: \"19930a0b-e995-434a-8bf1-d1040522618f\") " pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:01.779560 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.779525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ldkn\" (UniqueName: \"kubernetes.io/projected/5a7e40a2-2fcc-481a-84aa-5de5def8eba0-kube-api-access-9ldkn\") pod \"payload-pre-processing-7964457696-r9vh4\" (UID: \"5a7e40a2-2fcc-481a-84aa-5de5def8eba0\") " pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.915027 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.914983 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:01.940165 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:01.940128 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:02.067863 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:02.067803 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r9vh4"] Jul 07 15:54:02.070869 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:54:02.070839 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a7e40a2_2fcc_481a_84aa_5de5def8eba0.slice/crio-405d75256f677e23eb19cddbfec9a66a6f7a0359a8394d342ec66a40f813d12f WatchSource:0}: Error finding container 405d75256f677e23eb19cddbfec9a66a6f7a0359a8394d342ec66a40f813d12f: Status 404 returned error can't find the container with id 405d75256f677e23eb19cddbfec9a66a6f7a0359a8394d342ec66a40f813d12f Jul 07 15:54:02.072961 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:02.072941 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:54:02.097990 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:02.097963 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mv6k"] Jul 07 15:54:02.099008 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:54:02.098976 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19930a0b_e995_434a_8bf1_d1040522618f.slice/crio-9971341472f54e245a91684d7e7feb6e7f0db091a86ec5149a6260340f60139c WatchSource:0}: Error finding container 9971341472f54e245a91684d7e7feb6e7f0db091a86ec5149a6260340f60139c: Status 404 returned error can't find the container with id 9971341472f54e245a91684d7e7feb6e7f0db091a86ec5149a6260340f60139c Jul 07 15:54:02.807260 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:02.807223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" event={"ID":"5a7e40a2-2fcc-481a-84aa-5de5def8eba0","Type":"ContainerStarted","Data":"405d75256f677e23eb19cddbfec9a66a6f7a0359a8394d342ec66a40f813d12f"} Jul 07 15:54:02.808770 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:02.808739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" event={"ID":"19930a0b-e995-434a-8bf1-d1040522618f","Type":"ContainerStarted","Data":"9971341472f54e245a91684d7e7feb6e7f0db091a86ec5149a6260340f60139c"} Jul 07 15:54:03.814918 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.814880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" event={"ID":"5a7e40a2-2fcc-481a-84aa-5de5def8eba0","Type":"ContainerStarted","Data":"36a8a8949bbc284e98b717f3ab3f84c61747d709a01ab40478b7a46bfeb06deb"} Jul 07 15:54:03.815347 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.814997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:03.816499 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.816475 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" event={"ID":"19930a0b-e995-434a-8bf1-d1040522618f","Type":"ContainerStarted","Data":"8dd463e53757c8c78a5b25b7e5098dcdb237fae1aee66bf48b0bfdda45e25c0a"} Jul 07 15:54:03.816618 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.816546 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:03.836734 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.836676 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" podStartSLOduration=1.4975155949999999 podStartE2EDuration="2.836658964s" totalImagesPullingTime="1.339143369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:03.835072475 +0000 UTC m=+617.066138741" watchObservedRunningTime="2026-07-07 15:54:03.836658964 +0000 UTC m=+617.067725239" Jul 07 15:54:03.856951 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:03.856890 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" podStartSLOduration=1.543478782 podStartE2EDuration="2.856872973s" totalImagesPullingTime="1.313394191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:03.854778154 +0000 UTC m=+617.085844430" watchObservedRunningTime="2026-07-07 15:54:03.856872973 +0000 UTC m=+617.087939248" Jul 07 15:54:11.385410 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:11.385293 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:11.481306 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:11.481262 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:14.822531 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:14.822489 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" podUID="5a7e40a2-2fcc-481a-84aa-5de5def8eba0" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.40:9004: connect: connection refused" Jul 07 15:54:14.822993 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:14.822869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5mv6k" Jul 07 15:54:17.216063 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:17.216028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:24.487233 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:24.487189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:24.609597 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:24.609566 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:24.822178 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:24.822149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-r9vh4" Jul 07 15:54:37.058529 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:37.058498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:51.051628 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.051587 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm"] Jul 07 15:54:51.055619 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.055594 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.064739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.064717 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:54:51.065856 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.065837 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vvllp\"" Jul 07 15:54:51.065856 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.065851 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:54:51.066051 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.065928 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 15:54:51.100782 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.100746 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm"] Jul 07 15:54:51.112700 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.112862 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.112862 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdea585-c221-4107-8cb5-63128725b789-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.112862 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.112862 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.113056 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.112868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6glxh\" (UniqueName: \"kubernetes.io/projected/4bdea585-c221-4107-8cb5-63128725b789-kube-api-access-6glxh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.131751 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.131716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:51.213771 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdea585-c221-4107-8cb5-63128725b789-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.213953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.213953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.213953 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6glxh\" (UniqueName: \"kubernetes.io/projected/4bdea585-c221-4107-8cb5-63128725b789-kube-api-access-6glxh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.214111 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.214111 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.213986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.214345 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.214319 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.214345 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.214343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.214496 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.214446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.216143 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.216111 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdea585-c221-4107-8cb5-63128725b789-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.216340 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.216323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdea585-c221-4107-8cb5-63128725b789-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.226788 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.226766 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6glxh\" (UniqueName: \"kubernetes.io/projected/4bdea585-c221-4107-8cb5-63128725b789-kube-api-access-6glxh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jv6cm\" (UID: \"4bdea585-c221-4107-8cb5-63128725b789\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.365109 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.365013 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:54:51.392686 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.392655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:54:51.517081 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:51.517049 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm"] Jul 07 15:54:51.520765 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:54:51.520732 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdea585_c221_4107_8cb5_63128725b789.slice/crio-84140caa457aef58e7fc5985ac26f93bc1eadeb3a2d4326768760de597b88d29 WatchSource:0}: Error finding container 84140caa457aef58e7fc5985ac26f93bc1eadeb3a2d4326768760de597b88d29: Status 404 returned error can't find the container with id 84140caa457aef58e7fc5985ac26f93bc1eadeb3a2d4326768760de597b88d29 Jul 07 15:54:52.078453 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:52.078413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" event={"ID":"4bdea585-c221-4107-8cb5-63128725b789","Type":"ContainerStarted","Data":"84140caa457aef58e7fc5985ac26f93bc1eadeb3a2d4326768760de597b88d29"} Jul 07 15:54:53.085266 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:53.085223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" event={"ID":"4bdea585-c221-4107-8cb5-63128725b789","Type":"ContainerStarted","Data":"34934f581f60a2d8c07e9a68cd8883a0756bd3e3ed3c5c3e9456b41b28ef314d"} Jul 07 15:54:53.107534 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:54:53.107478 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" podStartSLOduration=1.008898537 podStartE2EDuration="2.107459725s" totalImagesPullingTime="1.098561188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:53.105878016 +0000 UTC m=+666.336944301" watchObservedRunningTime="2026-07-07 15:54:53.107459725 +0000 UTC m=+666.338526016" Jul 07 15:55:01.366022 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:01.365973 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:55:01.366572 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:01.366057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:55:01.378937 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:01.378907 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:55:02.072569 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:02.072531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:02.145135 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:02.145098 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jv6cm" Jul 07 15:55:09.229572 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.229534 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd"] Jul 07 15:55:09.233130 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.233111 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.236150 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.236131 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 15:55:09.244572 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.244549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd"] Jul 07 15:55:09.262230 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.262202 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:09.377569 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.377569 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d2d3d68-c079-4c16-b50e-e8a402544676-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.377790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlr6k\" (UniqueName: \"kubernetes.io/projected/1d2d3d68-c079-4c16-b50e-e8a402544676-kube-api-access-jlr6k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.377790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.377790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.377790 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.377722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478527 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478527 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478745 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478745 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478850 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d2d3d68-c079-4c16-b50e-e8a402544676-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478850 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlr6k\" (UniqueName: \"kubernetes.io/projected/1d2d3d68-c079-4c16-b50e-e8a402544676-kube-api-access-jlr6k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.478980 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.478958 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.479048 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.479030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.479112 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.479073 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.480892 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.480838 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d2d3d68-c079-4c16-b50e-e8a402544676-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.481296 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.481273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d2d3d68-c079-4c16-b50e-e8a402544676-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.491220 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.491197 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlr6k\" (UniqueName: \"kubernetes.io/projected/1d2d3d68-c079-4c16-b50e-e8a402544676-kube-api-access-jlr6k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-65lcd\" (UID: \"1d2d3d68-c079-4c16-b50e-e8a402544676\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.542737 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.542702 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:09.680984 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:09.680957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd"] Jul 07 15:55:09.682390 ip-10-0-141-42 kubenswrapper[2562]: W0707 15:55:09.682353 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2d3d68_c079_4c16_b50e_e8a402544676.slice/crio-19dcab047af08da5397cd01dd808f30d50022299f11583ac4dec38c52a6b32f5 WatchSource:0}: Error finding container 19dcab047af08da5397cd01dd808f30d50022299f11583ac4dec38c52a6b32f5: Status 404 returned error can't find the container with id 19dcab047af08da5397cd01dd808f30d50022299f11583ac4dec38c52a6b32f5 Jul 07 15:55:10.174716 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:10.174626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" event={"ID":"1d2d3d68-c079-4c16-b50e-e8a402544676","Type":"ContainerStarted","Data":"b52ebf2893f0c584af5feaa980f4fdf51a9c66a8dfcc178296c50aa8c7e72b0d"} Jul 07 15:55:10.174716 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:10.174683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" event={"ID":"1d2d3d68-c079-4c16-b50e-e8a402544676","Type":"ContainerStarted","Data":"19dcab047af08da5397cd01dd808f30d50022299f11583ac4dec38c52a6b32f5"} Jul 07 15:55:10.195739 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:10.195691 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" podStartSLOduration=1.000936896 podStartE2EDuration="1.195675826s" totalImagesPullingTime="194.73893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:55:10.194825028 +0000 UTC m=+683.425891325" watchObservedRunningTime="2026-07-07 15:55:10.195675826 +0000 UTC m=+683.426742100" Jul 07 15:55:13.929026 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:13.928986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:19.543269 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:19.543225 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:19.543797 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:19.543318 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:19.555833 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:19.555811 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:20.237160 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:20.237130 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-65lcd" Jul 07 15:55:22.306266 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:22.306229 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:27.989985 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:27.989943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:28.053627 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:28.053588 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:59.595616 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:59.595584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:55:59.663178 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:55:59.663142 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:56:36.084944 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:56:36.084912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:56:36.188066 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:56:36.188031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:56:37.259962 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:56:37.259917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:56:37.365655 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:56:37.365624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:57:24.071487 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:57:24.071403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:57:24.157291 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:57:24.157250 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:57:48.097110 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:57:48.097070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:57:49.672857 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:57:49.672817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:30.366734 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:30.366697 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:30.461912 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:30.461875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:30.626580 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:30.626460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:37.782531 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:37.782500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:37.880531 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:37.880493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:37.997988 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:37.997944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:38.189251 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:38.189218 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:44.456152 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:44.456079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:44.524415 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:44.524370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 15:58:47.379095 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:47.379066 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:58:47.379547 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:47.379202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 15:58:47.386793 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:47.386770 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:58:47.386972 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:58:47.386776 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 15:59:31.234824 ip-10-0-141-42 kubenswrapper[2562]: I0707 15:59:31.234739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:01:09.179410 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:01:09.179355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:03:19.164293 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:03:19.164204 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:03:47.414735 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:03:47.414704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:03:47.416669 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:03:47.416648 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:03:47.422060 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:03:47.422043 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:03:47.423978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:03:47.423959 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:07:01.418966 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:07:01.418929 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:07:24.408957 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:07:24.408921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:08:47.446670 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:08:47.446640 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:08:47.450056 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:08:47.450033 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:08:47.454589 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:08:47.454557 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:08:47.457903 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:08:47.457883 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:09:09.817306 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:09.817219 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:16.708734 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:16.708693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:24.958738 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:24.958700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:30.052908 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:30.052874 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:32.871214 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:32.871184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:32.974909 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:32.974870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:33.305482 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:33.305447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:39.322097 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.322040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:39.422532 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.422495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:39.508428 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.508389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:39.512256 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.512235 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.515042 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.515020 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-958428c6-openshift-default-dockercfg-xbpxg\"" Jul 07 16:09:39.533170 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.533137 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:39.544219 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqz6\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544434 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544434 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544581 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544581 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544581 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544759 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544759 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.544759 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.544675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.546870 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.546843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:39.643086 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.642994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:39.645143 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645114 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645646 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645646 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqz6\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645646 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645646 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645721 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.645818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.645755 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.646061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.646044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.647584 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.647560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.647703 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.647688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.654569 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.654544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.654920 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.654896 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqz6\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6\") pod \"e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.822232 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.822194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:39.979097 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.979065 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:39.980566 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:09:39.980537 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4f081a_804c_4234_b86d_98c6cee68cbb.slice/crio-adfb84a8a9c5e527c9c7c0b1c8c836b403ad6a5b0c4bd7dbacb004b982ee414c WatchSource:0}: Error finding container adfb84a8a9c5e527c9c7c0b1c8c836b403ad6a5b0c4bd7dbacb004b982ee414c: Status 404 returned error can't find the container with id adfb84a8a9c5e527c9c7c0b1c8c836b403ad6a5b0c4bd7dbacb004b982ee414c Jul 07 16:09:39.982767 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.982725 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:39.982853 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.982813 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:39.982899 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:39.982854 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:40.702209 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.702169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" event={"ID":"fe4f081a-804c-4234-b86d-98c6cee68cbb","Type":"ContainerStarted","Data":"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518"} Jul 07 16:09:40.702725 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.702234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" event={"ID":"fe4f081a-804c-4234-b86d-98c6cee68cbb","Type":"ContainerStarted","Data":"adfb84a8a9c5e527c9c7c0b1c8c836b403ad6a5b0c4bd7dbacb004b982ee414c"} Jul 07 16:09:40.712321 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.712293 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:40.727058 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.727017 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" podStartSLOduration=1.727002505 podStartE2EDuration="1.727002505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:40.72647737 +0000 UTC m=+1553.957543645" watchObservedRunningTime="2026-07-07 16:09:40.727002505 +0000 UTC m=+1553.958068780" Jul 07 16:09:40.808101 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.808061 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:40.822995 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.822963 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:40.878998 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:40.878951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:41.237842 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:41.237808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:41.412875 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:41.412838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:41.514977 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:41.514888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:41.827694 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:41.827662 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:42.712081 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:42.711962 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:42.723973 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:42.712917 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:43.577104 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:43.577067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:45.745404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:45.744869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:45.911002 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:45.910961 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:45.911209 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:45.911172 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" podUID="fe4f081a-804c-4234-b86d-98c6cee68cbb" containerName="istio-proxy" containerID="cri-o://03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518" gracePeriod=30 Jul 07 16:09:45.952267 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:45.952232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:46.047269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:46.047238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:46.111951 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:46.111899 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:50.085075 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:50.085027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:51.674925 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.674869 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:51.741579 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741544 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741579 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741585 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741631 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741650 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qqz6\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741668 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741756 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.741818 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741783 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.742114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741824 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token\") pod \"fe4f081a-804c-4234-b86d-98c6cee68cbb\" (UID: \"fe4f081a-804c-4234-b86d-98c6cee68cbb\") " Jul 07 16:09:51.742114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741856 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:09:51.742114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.741957 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:09:51.742114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:09:51.742318 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742157 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.742318 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742180 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.742318 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742182 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:09:51.742318 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742196 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.742792 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.742769 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:09:51.744170 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.744137 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:09:51.744276 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.744198 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6" (OuterVolumeSpecName: "kube-api-access-6qqz6") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "kube-api-access-6qqz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:09:51.744326 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.744289 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:09:51.744326 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.744292 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe4f081a-804c-4234-b86d-98c6cee68cbb" (UID: "fe4f081a-804c-4234-b86d-98c6cee68cbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:09:51.775250 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.775218 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe4f081a-804c-4234-b86d-98c6cee68cbb" containerID="03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518" exitCode=0 Jul 07 16:09:51.775418 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.775262 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" Jul 07 16:09:51.775418 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.775295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" event={"ID":"fe4f081a-804c-4234-b86d-98c6cee68cbb","Type":"ContainerDied","Data":"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518"} Jul 07 16:09:51.775418 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.775352 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh" event={"ID":"fe4f081a-804c-4234-b86d-98c6cee68cbb","Type":"ContainerDied","Data":"adfb84a8a9c5e527c9c7c0b1c8c836b403ad6a5b0c4bd7dbacb004b982ee414c"} Jul 07 16:09:51.775418 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.775390 2562 scope.go:117] "RemoveContainer" containerID="03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518" Jul 07 16:09:51.784841 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.784824 2562 scope.go:117] "RemoveContainer" containerID="03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518" Jul 07 16:09:51.785085 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:09:51.785064 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518\": container with ID starting with 03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518 not found: ID does not exist" containerID="03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518" Jul 07 16:09:51.785151 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.785092 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518"} err="failed to get container status \"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518\": rpc error: code = NotFound desc = could not find container \"03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518\": container with ID starting with 03619429536dcf43927c4655a161e43e761159f1c8738dc8596b42afd31ce518 not found: ID does not exist" Jul 07 16:09:51.797918 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.797888 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:51.801937 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.801914 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-958428c6-openshift-default-77d4dd6bf-4bjgh"] Jul 07 16:09:51.843098 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843071 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qqz6\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-kube-api-access-6qqz6\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.843227 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843102 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.843227 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843122 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4f081a-804c-4234-b86d-98c6cee68cbb-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.843227 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843133 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.843227 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843144 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.843227 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.843155 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4f081a-804c-4234-b86d-98c6cee68cbb-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:09:51.927334 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:51.927255 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe4f081a-804c-4234-b86d-98c6cee68cbb" path="/var/lib/kubelet/pods/fe4f081a-804c-4234-b86d-98c6cee68cbb/volumes" Jul 07 16:09:52.134006 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.133960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:52.236882 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.236782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:52.303456 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.303363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:09:52.303912 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.303888 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe4f081a-804c-4234-b86d-98c6cee68cbb" containerName="istio-proxy" Jul 07 16:09:52.303912 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.303913 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe4f081a-804c-4234-b86d-98c6cee68cbb" containerName="istio-proxy" Jul 07 16:09:52.304077 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.304003 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe4f081a-804c-4234-b86d-98c6cee68cbb" containerName="istio-proxy" Jul 07 16:09:52.308496 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.308478 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.310941 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.310922 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f8a48a23-openshift-default-dockercfg-72tzd\"" Jul 07 16:09:52.326097 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.326066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:09:52.334485 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.334452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:52.346735 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346735 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8fr5\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.346931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.347101 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.346975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.347101 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.347017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.435273 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.435220 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:52.447466 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447864 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8fr5\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447864 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.447864 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.447706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.448045 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.448017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.448107 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.448051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.448166 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.448135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.448269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.448243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.448458 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.448436 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.450038 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.450018 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.450122 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.450109 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.456983 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.456955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.457336 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.457312 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8fr5\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5\") pod \"e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.618974 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.618939 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:52.757893 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.757860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:09:52.759255 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:09:52.759229 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be29a3f_7aa7_4d10_956d_ea030579faef.slice/crio-377ed5f868ce8da864d6c1d0bf9ba43e211914c51e3e506342b48aaff68f4dbc WatchSource:0}: Error finding container 377ed5f868ce8da864d6c1d0bf9ba43e211914c51e3e506342b48aaff68f4dbc: Status 404 returned error can't find the container with id 377ed5f868ce8da864d6c1d0bf9ba43e211914c51e3e506342b48aaff68f4dbc Jul 07 16:09:52.761317 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.761284 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:52.761434 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.761361 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:52.761520 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.761435 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:09:52.785068 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:52.785033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" event={"ID":"8be29a3f-7aa7-4d10-956d-ea030579faef","Type":"ContainerStarted","Data":"377ed5f868ce8da864d6c1d0bf9ba43e211914c51e3e506342b48aaff68f4dbc"} Jul 07 16:09:53.450342 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.450302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:53.537825 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.537792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:53.603502 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.603469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:53.791992 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.791958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" event={"ID":"8be29a3f-7aa7-4d10-956d-ea030579faef","Type":"ContainerStarted","Data":"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9"} Jul 07 16:09:53.819434 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.819371 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" podStartSLOduration=1.8193561919999999 podStartE2EDuration="1.819356192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:09:53.817403796 +0000 UTC m=+1567.048470066" watchObservedRunningTime="2026-07-07 16:09:53.819356192 +0000 UTC m=+1567.050422467" Jul 07 16:09:53.864708 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:53.864673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:54.043293 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.043207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:54.147156 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.147122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:54.619303 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.619268 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:54.624455 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.624431 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:54.797046 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.797017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:54.798114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:54.798094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:09:55.502642 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:55.502608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:58.564149 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:58.564109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:09:58.902840 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:09:58.902745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:01.580352 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:01.580313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:01.681222 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:01.681189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:04.591708 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:04.591670 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:10:04.592178 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:04.591882 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerName="istio-proxy" containerID="cri-o://7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9" gracePeriod=30 Jul 07 16:10:04.614619 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:04.614580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:04.714522 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:04.714485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:06.823629 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:06.823592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:09.848424 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.848394 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:10:09.876083 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.875997 2562 generic.go:356] "Generic (PLEG): container finished" podID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerID="7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9" exitCode=0 Jul 07 16:10:09.876083 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.876033 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" Jul 07 16:10:09.876296 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.876081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" event={"ID":"8be29a3f-7aa7-4d10-956d-ea030579faef","Type":"ContainerDied","Data":"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9"} Jul 07 16:10:09.876296 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.876174 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" event={"ID":"8be29a3f-7aa7-4d10-956d-ea030579faef","Type":"ContainerDied","Data":"377ed5f868ce8da864d6c1d0bf9ba43e211914c51e3e506342b48aaff68f4dbc"} Jul 07 16:10:09.876296 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.876206 2562 scope.go:117] "RemoveContainer" containerID="7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9" Jul 07 16:10:09.885356 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.885337 2562 scope.go:117] "RemoveContainer" containerID="7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9" Jul 07 16:10:09.885663 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:10:09.885643 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9\": container with ID starting with 7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9 not found: ID does not exist" containerID="7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9" Jul 07 16:10:09.885735 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.885670 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9"} err="failed to get container status \"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9\": rpc error: code = NotFound desc = could not find container \"7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9\": container with ID starting with 7f436bf9b5ed4b94b910fc86eeaff56b6db2ba5a2d63457699279d1654c45fe9 not found: ID does not exist" Jul 07 16:10:09.899019 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.898990 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8fr5\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899098 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899052 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899136 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899107 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899170 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899148 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899213 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899194 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899450 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899334 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899450 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899411 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899616 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899487 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:09.899616 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899545 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899616 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899575 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs\") pod \"8be29a3f-7aa7-4d10-956d-ea030579faef\" (UID: \"8be29a3f-7aa7-4d10-956d-ea030579faef\") " Jul 07 16:10:09.899616 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899603 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:09.899834 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899724 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data" (OuterVolumeSpecName: "istio-data") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:09.899885 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899852 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:09.899939 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899880 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:10:09.899939 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899906 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:09.899939 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899927 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:09.900066 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.899941 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:09.901786 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.901761 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:10:09.901874 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.901791 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token" (OuterVolumeSpecName: "istio-token") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:09.901998 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.901982 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5" (OuterVolumeSpecName: "kube-api-access-g8fr5") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "kube-api-access-g8fr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:09.902149 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:09.902013 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8be29a3f-7aa7-4d10-956d-ea030579faef" (UID: "8be29a3f-7aa7-4d10-956d-ea030579faef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:10.001471 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001432 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8fr5\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-kube-api-access-g8fr5\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.001471 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001463 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.001471 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001474 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.001471 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001483 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.001767 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001491 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be29a3f-7aa7-4d10-956d-ea030579faef-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.001767 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.001498 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be29a3f-7aa7-4d10-956d-ea030579faef-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:10.193639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.193566 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:10:10.195677 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.195656 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg"] Jul 07 16:10:10.797222 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.797186 2562 patch_prober.go:28] interesting pod/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 16:10:10.797404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.797258 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f8a48a23-openshift-default-7cb8f5dbdf-48vmg" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 16:10:10.819676 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.819643 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:10.895598 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:10.895566 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:11.005151 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.005106 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:11.005503 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.005489 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerName="istio-proxy" Jul 07 16:10:11.005577 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.005505 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerName="istio-proxy" Jul 07 16:10:11.005623 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.005588 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" containerName="istio-proxy" Jul 07 16:10:11.008969 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.008947 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.011769 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.011742 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ce576197-openshift-default-dockercfg-tlmvg\"" Jul 07 16:10:11.023335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.023310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:11.051555 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.051459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:11.111676 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.111676 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.111894 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111713 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.111894 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchvn\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.111894 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.111894 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.112021 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.112021 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.112021 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.111963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.152209 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.152170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:11.213321 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213321 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213562 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213562 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213562 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213704 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213704 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213704 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchvn\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213867 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213867 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.213980 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213933 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.214030 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.213982 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.214168 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.214137 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.214246 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.214224 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.215784 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.215767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.215879 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.215863 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.223924 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.223889 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.224214 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.224196 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchvn\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn\") pod \"e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.320583 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.320479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:11.464770 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.464744 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:11.466335 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:10:11.466302 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10313a65_b579_4d42_ba71_29d4578c205c.slice/crio-08a214346a2c3080b569f64fe1ab69ecb311bba55e55de00cd87cac66f4746f3 WatchSource:0}: Error finding container 08a214346a2c3080b569f64fe1ab69ecb311bba55e55de00cd87cac66f4746f3: Status 404 returned error can't find the container with id 08a214346a2c3080b569f64fe1ab69ecb311bba55e55de00cd87cac66f4746f3 Jul 07 16:10:11.468505 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.468474 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:10:11.468578 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.468536 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:10:11.468578 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.468567 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:10:11.889371 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.889337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" event={"ID":"10313a65-b579-4d42-ba71-29d4578c205c","Type":"ContainerStarted","Data":"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6"} Jul 07 16:10:11.889594 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.889415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" event={"ID":"10313a65-b579-4d42-ba71-29d4578c205c","Type":"ContainerStarted","Data":"08a214346a2c3080b569f64fe1ab69ecb311bba55e55de00cd87cac66f4746f3"} Jul 07 16:10:11.912131 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.912068 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" podStartSLOduration=1.912052694 podStartE2EDuration="1.912052694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:10:11.910431677 +0000 UTC m=+1585.141497953" watchObservedRunningTime="2026-07-07 16:10:11.912052694 +0000 UTC m=+1585.143118968" Jul 07 16:10:11.928055 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:11.928017 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8be29a3f-7aa7-4d10-956d-ea030579faef" path="/var/lib/kubelet/pods/8be29a3f-7aa7-4d10-956d-ea030579faef/volumes" Jul 07 16:10:12.224776 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.224695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:12.307168 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.307114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:12.321006 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.320978 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:12.326044 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.326019 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:12.371664 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.371628 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:12.520438 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.520332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:12.880457 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.880416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:12.896401 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.896345 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:12.897199 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:12.897180 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:13.024332 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:13.024291 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:14.197989 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:14.197954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:16.582440 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.582398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:16.748112 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.748072 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:16.748360 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.748333 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" podUID="10313a65-b579-4d42-ba71-29d4578c205c" containerName="istio-proxy" containerID="cri-o://09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6" gracePeriod=30 Jul 07 16:10:16.780274 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.780240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:16.881833 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.881734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:16.951072 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:16.951038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:18.654351 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:18.654318 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:22.603564 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.603496 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:22.721728 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721694 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721728 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721735 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tchvn\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721757 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721772 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721790 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721822 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721869 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721944 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.721994 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.721970 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo\") pod \"10313a65-b579-4d42-ba71-29d4578c205c\" (UID: \"10313a65-b579-4d42-ba71-29d4578c205c\") " Jul 07 16:10:22.722451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722065 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:22.722451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722250 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.722451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722357 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:22.722451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722259 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:10:22.722451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722401 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:22.722972 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.722952 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data" (OuterVolumeSpecName: "istio-data") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:22.724142 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.724109 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:10:22.724257 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.724162 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:10:22.724523 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.724506 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn" (OuterVolumeSpecName: "kube-api-access-tchvn") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "kube-api-access-tchvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:22.724592 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.724519 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token" (OuterVolumeSpecName: "istio-token") pod "10313a65-b579-4d42-ba71-29d4578c205c" (UID: "10313a65-b579-4d42-ba71-29d4578c205c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:10:22.822716 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822677 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10313a65-b579-4d42-ba71-29d4578c205c-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822716 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822710 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tchvn\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-kube-api-access-tchvn\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822725 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822739 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10313a65-b579-4d42-ba71-29d4578c205c-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822752 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822762 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822773 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10313a65-b579-4d42-ba71-29d4578c205c-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.822935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.822787 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10313a65-b579-4d42-ba71-29d4578c205c-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:10:22.953981 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.953855 2562 generic.go:356] "Generic (PLEG): container finished" podID="10313a65-b579-4d42-ba71-29d4578c205c" containerID="09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6" exitCode=0 Jul 07 16:10:22.953981 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.953904 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" Jul 07 16:10:22.953981 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.953941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" event={"ID":"10313a65-b579-4d42-ba71-29d4578c205c","Type":"ContainerDied","Data":"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6"} Jul 07 16:10:22.954293 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.954007 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg" event={"ID":"10313a65-b579-4d42-ba71-29d4578c205c","Type":"ContainerDied","Data":"08a214346a2c3080b569f64fe1ab69ecb311bba55e55de00cd87cac66f4746f3"} Jul 07 16:10:22.954293 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.954039 2562 scope.go:117] "RemoveContainer" containerID="09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6" Jul 07 16:10:22.963977 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.963958 2562 scope.go:117] "RemoveContainer" containerID="09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6" Jul 07 16:10:22.964235 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:10:22.964217 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6\": container with ID starting with 09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6 not found: ID does not exist" containerID="09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6" Jul 07 16:10:22.964279 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.964243 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6"} err="failed to get container status \"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6\": rpc error: code = NotFound desc = could not find container \"09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6\": container with ID starting with 09814e554bda8279a82fba18ca01a2efa5e5684cc6e14eef19b42a4559ffb7b6 not found: ID does not exist" Jul 07 16:10:22.981923 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.981897 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:22.987371 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:22.987350 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ce576197-openshift-default-7cc84479bc-n6qvg"] Jul 07 16:10:23.402234 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:23.402196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:23.494742 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:23.494710 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:23.898955 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:23.898907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:23.926241 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:23.926209 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10313a65-b579-4d42-ba71-29d4578c205c" path="/var/lib/kubelet/pods/10313a65-b579-4d42-ba71-29d4578c205c/volumes" Jul 07 16:10:23.992354 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:23.992319 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:24.193215 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:24.193122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:25.454217 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:25.454181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:31.715954 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:31.715911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:31.901692 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:31.901656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:32.045795 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:32.045750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:32.465672 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:32.465569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:10:58.429628 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:10:58.429589 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:04.760087 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:04.760055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:04.829444 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:04.829414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:04.914608 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:04.914565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:20.865979 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:20.865943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:26.916160 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:26.916117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:27.018078 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:27.018039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:27.120970 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:27.120938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:37.676754 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:37.676715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:38.463874 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:38.463839 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:44.466715 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:44.466622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:44.586574 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:44.586526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:44.671751 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:44.671711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:44.772451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:44.772327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:45.562522 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:45.562474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:45.659161 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:45.659123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:45.774742 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:45.774707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:47.345345 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:47.345307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:53.347951 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:53.347912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:53.447913 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:53.447873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:11:54.080689 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:11:54.080655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.006952 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.006917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.118361 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.118315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.307244 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.307212 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.639931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.639850 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:00.640276 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.640258 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10313a65-b579-4d42-ba71-29d4578c205c" containerName="istio-proxy" Jul 07 16:12:00.640365 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.640279 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10313a65-b579-4d42-ba71-29d4578c205c" containerName="istio-proxy" Jul 07 16:12:00.640447 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.640394 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10313a65-b579-4d42-ba71-29d4578c205c" containerName="istio-proxy" Jul 07 16:12:00.643354 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.643333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.646288 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.646268 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-109c0ad8-openshift-default-dockercfg-8gr7p\"" Jul 07 16:12:00.662432 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.662395 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:00.671874 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.671840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.765466 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb2q9\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.765814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.765786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867204 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb2q9\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867422 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867422 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867530 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867530 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867614 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867614 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867614 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867747 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867850 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867825 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.867975 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.867950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.868054 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.868032 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.868211 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.868191 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.868316 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.868301 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.869899 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.869874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.870013 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.869874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.876411 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.876363 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.876501 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.876451 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb2q9\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9\") pod \"e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:00.903950 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.903859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:00.955038 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:00.955005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:01.102012 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.101985 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:01.103739 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:12:01.103710 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bdf7e5_579b_4f02_bda1_5bf336785339.slice/crio-f4effe105b7ad763594d90e13f3c707e8ed723a599f4d8d883009de9bf4ceb41 WatchSource:0}: Error finding container f4effe105b7ad763594d90e13f3c707e8ed723a599f4d8d883009de9bf4ceb41: Status 404 returned error can't find the container with id f4effe105b7ad763594d90e13f3c707e8ed723a599f4d8d883009de9bf4ceb41 Jul 07 16:12:01.105963 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.105929 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:01.106068 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.106013 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:01.106131 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.106063 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:01.471778 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.471689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerStarted","Data":"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043"} Jul 07 16:12:01.471778 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.471749 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerStarted","Data":"f4effe105b7ad763594d90e13f3c707e8ed723a599f4d8d883009de9bf4ceb41"} Jul 07 16:12:01.496545 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.496490 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podStartSLOduration=1.496474074 podStartE2EDuration="1.496474074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:12:01.493452077 +0000 UTC m=+1694.724518373" watchObservedRunningTime="2026-07-07 16:12:01.496474074 +0000 UTC m=+1694.727540348" Jul 07 16:12:01.956050 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:01.956013 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:02.960956 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:02.960921 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:03.482435 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:03.482407 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:03.483507 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:03.483485 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:03.590139 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:03.590108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:05.103396 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:05.103321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:08.131878 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:08.131843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:08.879581 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:08.879534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:08.979536 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:08.979502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:09.312301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:09.312266 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:15.425709 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:15.425673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:15.499655 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:15.499615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:15.859095 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:15.859059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:16.338129 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.338091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:16.438980 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.438949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:16.560338 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.560303 2562 generic.go:356] "Generic (PLEG): container finished" podID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerID="2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043" exitCode=0 Jul 07 16:12:16.560338 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.560344 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerDied","Data":"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043"} Jul 07 16:12:16.560770 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.560756 2562 scope.go:117] "RemoveContainer" containerID="2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043" Jul 07 16:12:16.561838 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.561805 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:16.561950 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.561875 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:16.561950 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:16.561907 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:17.567651 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:17.567618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerStarted","Data":"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822"} Jul 07 16:12:17.955876 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:17.955772 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:17.956783 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:17.956761 2562 patch_prober.go:28] interesting pod/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:12:17.956841 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:17.956810 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:12:18.483086 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:18.483045 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:18.956348 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:18.956307 2562 patch_prober.go:28] interesting pod/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:12:18.956781 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:18.956366 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:12:19.956241 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:19.956203 2562 patch_prober.go:28] interesting pod/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:12:19.956469 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:19.956262 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:12:20.955955 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:20.955917 2562 patch_prober.go:28] interesting pod/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 07 16:12:20.956149 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:20.955980 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 07 16:12:21.959846 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:21.959811 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:21.960639 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:21.960620 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:22.477051 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:22.477016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:22.577459 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:22.577421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:22.830830 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:22.830798 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:22.855828 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:22.855792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:23.196991 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:23.196893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:23.297711 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:23.297679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:23.437026 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:23.436989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:23.599129 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:23.599094 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" containerID="cri-o://60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822" gracePeriod=30 Jul 07 16:12:28.853166 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.853141 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:28.925814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925717 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.925814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925775 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925835 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925858 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925919 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925961 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.925997 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926036 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:28.926301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926167 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926174 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:28.926301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926215 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb2q9\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9\") pod \"e7bdf7e5-579b-4f02-bda1-5bf336785339\" (UID: \"e7bdf7e5-579b-4f02-bda1-5bf336785339\") " Jul 07 16:12:28.926301 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926290 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:28.926652 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926532 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:12:28.926652 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926589 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:28.926652 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926611 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:28.926652 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926626 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:28.926877 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.926853 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:28.928264 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.928234 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:12:28.928465 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.928444 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:12:28.928754 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.928727 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:12:28.928876 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:28.928727 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9" (OuterVolumeSpecName: "kube-api-access-jb2q9") pod "e7bdf7e5-579b-4f02-bda1-5bf336785339" (UID: "e7bdf7e5-579b-4f02-bda1-5bf336785339"). InnerVolumeSpecName "kube-api-access-jb2q9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:12:29.027403 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027342 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.027403 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027406 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.027619 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027418 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7bdf7e5-579b-4f02-bda1-5bf336785339-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.027619 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027426 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.027619 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027437 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7bdf7e5-579b-4f02-bda1-5bf336785339-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.027619 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.027445 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb2q9\" (UniqueName: \"kubernetes.io/projected/e7bdf7e5-579b-4f02-bda1-5bf336785339-kube-api-access-jb2q9\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:12:29.632960 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.632925 2562 generic.go:356] "Generic (PLEG): container finished" podID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerID="60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822" exitCode=0 Jul 07 16:12:29.633143 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.632981 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" Jul 07 16:12:29.633143 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.633018 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerDied","Data":"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822"} Jul 07 16:12:29.633143 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.633085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz" event={"ID":"e7bdf7e5-579b-4f02-bda1-5bf336785339","Type":"ContainerDied","Data":"f4effe105b7ad763594d90e13f3c707e8ed723a599f4d8d883009de9bf4ceb41"} Jul 07 16:12:29.633143 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.633114 2562 scope.go:117] "RemoveContainer" containerID="60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822" Jul 07 16:12:29.646591 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.646438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:29.646656 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.646626 2562 scope.go:117] "RemoveContainer" containerID="2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043" Jul 07 16:12:29.654838 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.654817 2562 scope.go:117] "RemoveContainer" containerID="60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822" Jul 07 16:12:29.655127 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:12:29.655107 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822\": container with ID starting with 60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822 not found: ID does not exist" containerID="60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822" Jul 07 16:12:29.655200 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.655141 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822"} err="failed to get container status \"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822\": rpc error: code = NotFound desc = could not find container \"60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822\": container with ID starting with 60d76b030d8e77b1d8446f9bef1ec9f1e1ed9dc476b48a26e61b0c6798e15822 not found: ID does not exist" Jul 07 16:12:29.655200 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.655168 2562 scope.go:117] "RemoveContainer" containerID="2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043" Jul 07 16:12:29.655441 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:12:29.655423 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043\": container with ID starting with 2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043 not found: ID does not exist" containerID="2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043" Jul 07 16:12:29.655486 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.655447 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043"} err="failed to get container status \"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043\": rpc error: code = NotFound desc = could not find container \"2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043\": container with ID starting with 2097e9954f7ea4ee102b2791ef5a8c7651367f4238ee1b90194b56c1e43c8043 not found: ID does not exist" Jul 07 16:12:29.659789 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.659764 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:29.667395 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.667357 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-109c0ad8-openshift-default-7b9fb946ff-9rngz"] Jul 07 16:12:29.746423 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.746342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:29.842974 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.842938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:29.926688 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.926592 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" path="/var/lib/kubelet/pods/e7bdf7e5-579b-4f02-bda1-5bf336785339/volumes" Jul 07 16:12:29.994892 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:29.994858 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:36.163074 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:36.163040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:36.273185 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:36.273143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:42.997349 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:42.997308 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:43.271463 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.271358 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:12:43.271745 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.271730 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:12:43.271794 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.271746 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:12:43.271838 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.271811 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:12:43.271838 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.271821 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:12:43.276325 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.276302 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.279260 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.279242 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cadb2345-openshift-default-dockercfg-g2kg8\"" Jul 07 16:12:43.288800 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.288769 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:12:43.300184 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.300153 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:43.356012 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.355979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356012 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dcw\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.356282 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.356269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.456923 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.456882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.456923 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.456921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.456943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.456967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.456990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dcw\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457562 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457400 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457562 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457768 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.457826 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.457796 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.458117 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.458093 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.459366 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.459347 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.459799 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.459781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.465892 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.465867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.466013 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.465938 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dcw\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw\") pod \"e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.552063 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.552031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:43.587966 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.587928 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:43.727917 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.727889 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:12:43.729634 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:12:43.729605 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8defba_4531_4856_8057_b62f3f6d06a1.slice/crio-f424c58eabb0f279d1d6597389247f68c1a507580f1476a42ac3569684692a1d WatchSource:0}: Error finding container f424c58eabb0f279d1d6597389247f68c1a507580f1476a42ac3569684692a1d: Status 404 returned error can't find the container with id f424c58eabb0f279d1d6597389247f68c1a507580f1476a42ac3569684692a1d Jul 07 16:12:43.731778 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.731743 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:43.731848 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.731813 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:43.731885 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:43.731846 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:44.713035 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:44.712999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerStarted","Data":"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1"} Jul 07 16:12:44.713448 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:44.713055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerStarted","Data":"f424c58eabb0f279d1d6597389247f68c1a507580f1476a42ac3569684692a1d"} Jul 07 16:12:44.779396 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:44.779336 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podStartSLOduration=1.779321242 podStartE2EDuration="1.779321242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:12:44.776932741 +0000 UTC m=+1738.007999018" watchObservedRunningTime="2026-07-07 16:12:44.779321242 +0000 UTC m=+1738.010387517" Jul 07 16:12:45.588390 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:45.588347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:45.593263 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:45.593236 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:45.718091 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:45.718057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:45.719021 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:45.719002 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:12:46.689105 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:46.689055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:46.773589 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:46.773552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:46.848247 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:46.848212 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:46.999875 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:46.999780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:47.383191 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:47.383156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:47.481821 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:47.481788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:51.190744 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:51.190702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:12:59.795053 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.795019 2562 generic.go:356] "Generic (PLEG): container finished" podID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerID="80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1" exitCode=0 Jul 07 16:12:59.795488 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.795091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerDied","Data":"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1"} Jul 07 16:12:59.795532 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.795506 2562 scope.go:117] "RemoveContainer" containerID="80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1" Jul 07 16:12:59.796519 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.796489 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:59.796600 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.796560 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:12:59.796600 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:12:59.796592 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:13:00.718205 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:00.718160 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:13:00.802062 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:00.802031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerStarted","Data":"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01"} Jul 07 16:13:01.588693 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:01.588639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:13:01.590020 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:01.589993 2562 patch_prober.go:28] interesting pod/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:13:01.590156 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:01.590074 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:13:02.588879 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:02.588841 2562 patch_prober.go:28] interesting pod/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:13:02.589279 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:02.588898 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:13:03.589219 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:03.589173 2562 patch_prober.go:28] interesting pod/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:13:03.589662 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:03.589241 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:13:04.589000 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:04.588960 2562 patch_prober.go:28] interesting pod/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:13:04.589188 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:04.589049 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:13:05.593065 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:05.593037 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:13:05.593469 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:05.593400 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:13:05.594009 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:05.593993 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:13:11.831120 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:11.831084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:11.930869 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:11.930836 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:16.845206 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:16.845162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:38.992645 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:38.992609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:39.527272 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:39.527234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:39.630653 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:39.630617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:39.780647 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:39.780560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:45.090395 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:45.090344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:13:47.481672 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:47.481637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:13:47.485775 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:47.485748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:13:47.489206 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:47.489181 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:13:47.493231 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:47.493213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:13:50.160893 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:13:50.160855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:27.000681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:27.000647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:28.036575 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:28.036535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:34.165408 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:34.165356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:34.265166 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:34.265133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:34.439183 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:34.439084 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:14:34.439361 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:34.439289 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" containerID="cri-o://b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01" gracePeriod=30 Jul 07 16:14:34.454730 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:34.454696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:35.596459 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:35.596420 2562 patch_prober.go:28] interesting pod/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 16:14:35.596847 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:35.596492 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 16:14:35.746440 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:35.746406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:35.839544 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:35.839501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:35.943267 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:35.943167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:36.647019 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:36.646984 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:36.747524 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:36.747481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:36.846873 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:36.846842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:38.209705 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:38.209667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:39.696369 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.696346 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:14:39.811249 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811220 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811499 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811264 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811499 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811426 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811499 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811483 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7dcw\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811650 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811519 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811650 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811544 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811650 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811577 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811650 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811578 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:39.811862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811754 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo\") pod \"bf8defba-4531-4856-8057-b62f3f6d06a1\" (UID: \"bf8defba-4531-4856-8057-b62f3f6d06a1\") " Jul 07 16:14:39.811862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811834 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:39.811862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811737 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:14:39.812081 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.811865 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:39.812081 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.812072 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf8defba-4531-4856-8057-b62f3f6d06a1-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.812187 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.812090 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.812187 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.812103 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.812187 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.812115 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.812338 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.812202 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:39.813752 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.813719 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:14:39.813868 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.813811 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:14:39.814116 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.814099 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:14:39.814116 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.814103 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw" (OuterVolumeSpecName: "kube-api-access-g7dcw") pod "bf8defba-4531-4856-8057-b62f3f6d06a1" (UID: "bf8defba-4531-4856-8057-b62f3f6d06a1"). InnerVolumeSpecName "kube-api-access-g7dcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:14:39.912933 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.912899 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.912933 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.912926 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.912933 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.912935 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.912933 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.912944 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf8defba-4531-4856-8057-b62f3f6d06a1-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:39.913201 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:39.912953 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7dcw\" (UniqueName: \"kubernetes.io/projected/bf8defba-4531-4856-8057-b62f3f6d06a1-kube-api-access-g7dcw\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:14:40.337611 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.337569 2562 generic.go:356] "Generic (PLEG): container finished" podID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerID="b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01" exitCode=0 Jul 07 16:14:40.337799 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.337622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerDied","Data":"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01"} Jul 07 16:14:40.337799 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.337647 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" Jul 07 16:14:40.337799 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.337691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6" event={"ID":"bf8defba-4531-4856-8057-b62f3f6d06a1","Type":"ContainerDied","Data":"f424c58eabb0f279d1d6597389247f68c1a507580f1476a42ac3569684692a1d"} Jul 07 16:14:40.337799 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.337723 2562 scope.go:117] "RemoveContainer" containerID="b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01" Jul 07 16:14:40.346813 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.346793 2562 scope.go:117] "RemoveContainer" containerID="80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1" Jul 07 16:14:40.354688 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.354623 2562 scope.go:117] "RemoveContainer" containerID="b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01" Jul 07 16:14:40.354929 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:14:40.354909 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01\": container with ID starting with b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01 not found: ID does not exist" containerID="b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01" Jul 07 16:14:40.354995 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.354937 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01"} err="failed to get container status \"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01\": rpc error: code = NotFound desc = could not find container \"b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01\": container with ID starting with b5ba9476a0de16bf8f400452071af1c70e5f8df09613679b25104a9efc315a01 not found: ID does not exist" Jul 07 16:14:40.354995 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.354956 2562 scope.go:117] "RemoveContainer" containerID="80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1" Jul 07 16:14:40.355173 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:14:40.355157 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1\": container with ID starting with 80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1 not found: ID does not exist" containerID="80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1" Jul 07 16:14:40.355209 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.355179 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1"} err="failed to get container status \"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1\": rpc error: code = NotFound desc = could not find container \"80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1\": container with ID starting with 80f79611c9a25196003d59e037e4934d25d24d4b0c70b2d378c95a68b2bd2ab1 not found: ID does not exist" Jul 07 16:14:40.360729 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.360706 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:14:40.364074 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:40.364052 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cadb2345-openshift-default-54dfc5f755-fbcd6"] Jul 07 16:14:41.926359 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:41.926319 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" path="/var/lib/kubelet/pods/bf8defba-4531-4856-8057-b62f3f6d06a1/volumes" Jul 07 16:14:44.359512 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:44.359472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:44.479655 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:44.479620 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:45.045309 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:45.045271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:48.572955 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:48.572918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:51.519657 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:51.519619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:51.635748 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:51.635719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:14:51.763470 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:14:51.763433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:00.153564 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153526 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153888 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153900 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153911 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153916 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153929 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.153935 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7bdf7e5-579b-4f02-bda1-5bf336785339" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.154007 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.154061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.154020 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8defba-4531-4856-8057-b62f3f6d06a1" containerName="istio-proxy" Jul 07 16:15:00.157029 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.157006 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:15:00.159902 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.159883 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 07 16:15:00.161030 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.161013 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7b4t8\"" Jul 07 16:15:00.161126 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.161026 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 07 16:15:00.165154 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.165132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:15:00.295232 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.295183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gl89\" (UniqueName: \"kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89\") pod \"maas-api-key-cleanup-29724015-5xb92\" (UID: \"79dacaad-3eac-48a3-9b0e-25972b272b25\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:15:00.396085 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.396048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gl89\" (UniqueName: \"kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89\") pod \"maas-api-key-cleanup-29724015-5xb92\" (UID: \"79dacaad-3eac-48a3-9b0e-25972b272b25\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:15:00.405780 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.405707 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gl89\" (UniqueName: \"kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89\") pod \"maas-api-key-cleanup-29724015-5xb92\" (UID: \"79dacaad-3eac-48a3-9b0e-25972b272b25\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:15:00.468934 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.468895 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:15:00.604829 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.604799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:15:00.606676 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:15:00.606642 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79dacaad_3eac_48a3_9b0e_25972b272b25.slice/crio-b7adb86fb8f7c5bc25713e1a1b0a7530bb37660fdf781a95c5b782ad4324ffa9 WatchSource:0}: Error finding container b7adb86fb8f7c5bc25713e1a1b0a7530bb37660fdf781a95c5b782ad4324ffa9: Status 404 returned error can't find the container with id b7adb86fb8f7c5bc25713e1a1b0a7530bb37660fdf781a95c5b782ad4324ffa9 Jul 07 16:15:00.608736 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:00.608717 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:15:01.450823 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:01.450790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" event={"ID":"79dacaad-3eac-48a3-9b0e-25972b272b25","Type":"ContainerStarted","Data":"b7adb86fb8f7c5bc25713e1a1b0a7530bb37660fdf781a95c5b782ad4324ffa9"} Jul 07 16:15:02.337134 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:02.337091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:02.757785 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:02.757681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:03.462837 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:03.462793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" event={"ID":"79dacaad-3eac-48a3-9b0e-25972b272b25","Type":"ContainerStarted","Data":"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8"} Jul 07 16:15:03.481393 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:03.481341 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" podStartSLOduration=0.917587968 podStartE2EDuration="3.481324276s" totalImagesPullingTime="2.563736308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:03.48088522 +0000 UTC m=+1876.711951497" watchObservedRunningTime="2026-07-07 16:15:03.481324276 +0000 UTC m=+1876.712390551" Jul 07 16:15:23.692522 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:23.692482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:27.501966 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:27.501931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:27.654595 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:27.654555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:29.855235 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:29.855196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:29.946975 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:29.946937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:30.220048 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.219956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:30.566080 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.566046 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:15:30.569609 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.569583 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.574247 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.574229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-be8b55-openshift-default-dockercfg-jgdgs\"" Jul 07 16:15:30.591357 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.591336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:15:30.609464 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.609445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:30.669911 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.669872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.669911 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.669911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.669930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.669978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.670012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.670084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdq92\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670114 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.670111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670278 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.670145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.670278 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.670188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771445 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdq92\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771445 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.771722 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.772112 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.771882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.772112 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.772053 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.772112 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.772086 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.772223 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.772135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.772461 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.772440 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.773909 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.773886 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.774231 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.774211 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.781606 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.781564 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdq92\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.785263 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.785242 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token\") pod \"e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.880019 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.879934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:30.895432 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:30.895401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:31.032315 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.032288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:15:31.034336 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:15:31.034310 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ff808c_9391_4fa1_9606_ead4078651dd.slice/crio-2def15cbc22a3ff0130625dcd8443f82cc94b10bab9430acde777d22def9a928 WatchSource:0}: Error finding container 2def15cbc22a3ff0130625dcd8443f82cc94b10bab9430acde777d22def9a928: Status 404 returned error can't find the container with id 2def15cbc22a3ff0130625dcd8443f82cc94b10bab9430acde777d22def9a928 Jul 07 16:15:31.036366 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.036333 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:31.036458 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.036428 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:31.036505 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.036456 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:31.608860 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.608821 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerStarted","Data":"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed"} Jul 07 16:15:31.609082 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.608883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerStarted","Data":"2def15cbc22a3ff0130625dcd8443f82cc94b10bab9430acde777d22def9a928"} Jul 07 16:15:31.638910 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.638853 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podStartSLOduration=1.6388391100000002 podStartE2EDuration="1.63883911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:15:31.636880655 +0000 UTC m=+1904.867946940" watchObservedRunningTime="2026-07-07 16:15:31.63883911 +0000 UTC m=+1904.869905436" Jul 07 16:15:31.880443 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:31.880322 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:32.885991 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:32.885959 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:33.593126 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.593087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:33.619537 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.619501 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:33.620578 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.620557 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:33.695683 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.695651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:33.793929 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.793885 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:33.900019 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:33.899921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:34.314530 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:34.314500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:34.494516 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:34.494484 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:38.128236 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:38.128179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:38.462873 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:38.462779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:38.779661 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:38.779561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:38.957090 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:38.957059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:41.781130 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:41.781089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:41.868523 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:41.868486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:42.001624 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:42.001572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:42.526026 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:42.525992 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:42.641606 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:42.641568 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:42.746935 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:42.746900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:46.290612 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:46.290571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:48.621120 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.621030 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 07 16:15:48.621633 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.621108 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 07 16:15:48.706071 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.705983 2562 generic.go:356] "Generic (PLEG): container finished" podID="48ff808c-9391-4fa1-9606-ead4078651dd" containerID="aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed" exitCode=0 Jul 07 16:15:48.706071 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.706061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerDied","Data":"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed"} Jul 07 16:15:48.706568 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.706549 2562 scope.go:117] "RemoveContainer" containerID="aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed" Jul 07 16:15:48.707635 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.707603 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:48.707700 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.707675 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:48.707742 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:48.707708 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:15:49.713160 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:49.713122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerStarted","Data":"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7"} Jul 07 16:15:49.880794 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:49.880755 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:49.881018 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:49.880994 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 07 16:15:49.881065 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:49.881046 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 07 16:15:50.253062 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:50.253027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:50.881182 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:50.881139 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 07 16:15:50.881713 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:50.881205 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 07 16:15:51.881007 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:51.880967 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 07 16:15:51.881190 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:51.881029 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 07 16:15:52.880576 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:52.880541 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 07 16:15:52.880985 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:52.880610 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 07 16:15:53.885252 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:53.885219 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:53.885721 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:53.885617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:53.886364 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:53.886347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:15:55.204996 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:55.204963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:15:55.312913 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:15:55.312834 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:00.012577 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:00.012540 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:16:00.013058 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:00.012759 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" podUID="79dacaad-3eac-48a3-9b0e-25972b272b25" containerName="cleanup" containerID="cri-o://db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8" gracePeriod=30 Jul 07 16:16:03.260314 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:03.260275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:03.361601 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:03.361560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:09.749389 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:09.749338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:30.196306 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.196278 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:16:30.310341 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.310305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gl89\" (UniqueName: \"kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89\") pod \"79dacaad-3eac-48a3-9b0e-25972b272b25\" (UID: \"79dacaad-3eac-48a3-9b0e-25972b272b25\") " Jul 07 16:16:30.312609 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.312583 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89" (OuterVolumeSpecName: "kube-api-access-6gl89") pod "79dacaad-3eac-48a3-9b0e-25972b272b25" (UID: "79dacaad-3eac-48a3-9b0e-25972b272b25"). InnerVolumeSpecName "kube-api-access-6gl89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:16:30.411865 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.411829 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gl89\" (UniqueName: \"kubernetes.io/projected/79dacaad-3eac-48a3-9b0e-25972b272b25-kube-api-access-6gl89\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:16:30.941168 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.941129 2562 generic.go:356] "Generic (PLEG): container finished" podID="79dacaad-3eac-48a3-9b0e-25972b272b25" containerID="db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8" exitCode=137 Jul 07 16:16:30.941168 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.941171 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" Jul 07 16:16:30.941404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.941214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" event={"ID":"79dacaad-3eac-48a3-9b0e-25972b272b25","Type":"ContainerDied","Data":"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8"} Jul 07 16:16:30.941404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.941265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92" event={"ID":"79dacaad-3eac-48a3-9b0e-25972b272b25","Type":"ContainerDied","Data":"b7adb86fb8f7c5bc25713e1a1b0a7530bb37660fdf781a95c5b782ad4324ffa9"} Jul 07 16:16:30.941404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.941296 2562 scope.go:117] "RemoveContainer" containerID="db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8" Jul 07 16:16:30.950579 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.950557 2562 scope.go:117] "RemoveContainer" containerID="db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8" Jul 07 16:16:30.950822 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:16:30.950802 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8\": container with ID starting with db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8 not found: ID does not exist" containerID="db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8" Jul 07 16:16:30.950904 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.950832 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8"} err="failed to get container status \"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8\": rpc error: code = NotFound desc = could not find container \"db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8\": container with ID starting with db10b50ef02d9bf9fc00c32bf5cd25a512c7d0b8bcd73f49dfc92cadaaddf3d8 not found: ID does not exist" Jul 07 16:16:30.965009 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.964985 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:16:30.968709 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:30.968687 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724015-5xb92"] Jul 07 16:16:31.926463 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:31.926428 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79dacaad-3eac-48a3-9b0e-25972b272b25" path="/var/lib/kubelet/pods/79dacaad-3eac-48a3-9b0e-25972b272b25/volumes" Jul 07 16:16:34.711482 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:34.711449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:35.619406 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:35.619342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:35.764512 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:35.764476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:36.169694 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:36.169655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:36.268687 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:36.268652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:36.369476 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:36.369437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:36.460892 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:36.460797 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:16:41.250442 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:16:41.250332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:07.587489 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:07.587449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:18.896222 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:18.896182 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:19.263287 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:19.263184 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:17:19.263505 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:19.263458 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" containerID="cri-o://db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7" gracePeriod=30 Jul 07 16:17:19.270757 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:19.270725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:19.746902 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:19.746866 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:20.201274 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:20.201230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:20.519613 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:20.519516 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:20.627830 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:20.627790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:20.983048 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:20.983013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:23.888022 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:23.887982 2562 patch_prober.go:28] interesting pod/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 16:17:23.888413 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:23.888046 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 16:17:24.525161 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.525097 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:17:24.589960 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.589922 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.589986 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590023 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590039 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdq92\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590066 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590102 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590144 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590498 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590205 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590498 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590228 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data\") pod \"48ff808c-9391-4fa1-9606-ead4078651dd\" (UID: \"48ff808c-9391-4fa1-9606-ead4078651dd\") " Jul 07 16:17:24.590498 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590323 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:17:24.590660 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590566 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:17:24.590715 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590676 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ff808c-9391-4fa1-9606-ead4078651dd-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.590715 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590698 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.590841 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590809 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:17:24.590922 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.590896 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:17:24.591313 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.591288 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:17:24.592451 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.592421 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:17:24.592764 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.592734 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:17:24.592861 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.592791 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:17:24.592861 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.592794 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92" (OuterVolumeSpecName: "kube-api-access-xdq92") pod "48ff808c-9391-4fa1-9606-ead4078651dd" (UID: "48ff808c-9391-4fa1-9606-ead4078651dd"). InnerVolumeSpecName "kube-api-access-xdq92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:17:24.691733 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691696 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691733 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691728 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ff808c-9391-4fa1-9606-ead4078651dd-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691733 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691738 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdq92\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-kube-api-access-xdq92\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691750 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691759 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691767 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ff808c-9391-4fa1-9606-ead4078651dd-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:24.691978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:24.691775 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ff808c-9391-4fa1-9606-ead4078651dd-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:17:25.234432 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.234373 2562 generic.go:356] "Generic (PLEG): container finished" podID="48ff808c-9391-4fa1-9606-ead4078651dd" containerID="db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7" exitCode=0 Jul 07 16:17:25.234922 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.234456 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" Jul 07 16:17:25.234922 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.234471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerDied","Data":"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7"} Jul 07 16:17:25.234922 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.234531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8" event={"ID":"48ff808c-9391-4fa1-9606-ead4078651dd","Type":"ContainerDied","Data":"2def15cbc22a3ff0130625dcd8443f82cc94b10bab9430acde777d22def9a928"} Jul 07 16:17:25.234922 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.234555 2562 scope.go:117] "RemoveContainer" containerID="db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7" Jul 07 16:17:25.244139 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.244120 2562 scope.go:117] "RemoveContainer" containerID="aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed" Jul 07 16:17:25.253118 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.253101 2562 scope.go:117] "RemoveContainer" containerID="db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7" Jul 07 16:17:25.253434 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:17:25.253368 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7\": container with ID starting with db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7 not found: ID does not exist" containerID="db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7" Jul 07 16:17:25.253486 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.253441 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7"} err="failed to get container status \"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7\": rpc error: code = NotFound desc = could not find container \"db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7\": container with ID starting with db71509cfaf16ce06ed5589d844266a216bbf05cea9416170352a50fdbd2b1a7 not found: ID does not exist" Jul 07 16:17:25.253486 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.253459 2562 scope.go:117] "RemoveContainer" containerID="aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed" Jul 07 16:17:25.253682 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:17:25.253668 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed\": container with ID starting with aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed not found: ID does not exist" containerID="aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed" Jul 07 16:17:25.253721 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.253685 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed"} err="failed to get container status \"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed\": rpc error: code = NotFound desc = could not find container \"aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed\": container with ID starting with aa941f26d6a06063be335a122cdf2fd11c6270efd1e39eeda0f5e42ae19263ed not found: ID does not exist" Jul 07 16:17:25.302122 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.302091 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:17:25.312728 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.312700 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-be8b55-openshift-default-566bb7bcbc-hlsv8"] Jul 07 16:17:25.926147 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:25.926112 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" path="/var/lib/kubelet/pods/48ff808c-9391-4fa1-9606-ead4078651dd/volumes" Jul 07 16:17:28.383021 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:28.382988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:28.496978 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:28.496936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:28.873525 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:28.873480 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:29.410580 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.410547 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:29.819335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819294 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:17:29.819742 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819724 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79dacaad-3eac-48a3-9b0e-25972b272b25" containerName="cleanup" Jul 07 16:17:29.819845 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819745 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79dacaad-3eac-48a3-9b0e-25972b272b25" containerName="cleanup" Jul 07 16:17:29.819845 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819764 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:17:29.819845 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819772 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:17:29.819992 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819872 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:17:29.819992 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819889 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:17:29.819992 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.819900 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79dacaad-3eac-48a3-9b0e-25972b272b25" containerName="cleanup" Jul 07 16:17:29.824429 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.824407 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.827297 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.827275 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1dcebd-openshift-default-dockercfg-jqvb8\"" Jul 07 16:17:29.861502 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.861466 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:17:29.890053 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.890023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:29.943976 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.943931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.943984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqllc\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:29.944411 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:29.944196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.010977 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.010942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:30.045229 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045450 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045450 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045618 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllc\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045618 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045618 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045804 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045804 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045804 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045961 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045863 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.045961 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.045919 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.046070 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.046016 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.046070 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.046023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.046167 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.046120 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.047832 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.047812 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.047972 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.047956 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.056322 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.056292 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqllc\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.056846 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.056826 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token\") pod \"e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.135321 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.135221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:30.289005 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.288980 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:17:30.291529 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:17:30.291505 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb34c60_bac2_446b_8294_1251390811b8.slice/crio-58e36738555dcba00b275f4fac6d80319ad5f80f11e17d21e59f5cff409358a7 WatchSource:0}: Error finding container 58e36738555dcba00b275f4fac6d80319ad5f80f11e17d21e59f5cff409358a7: Status 404 returned error can't find the container with id 58e36738555dcba00b275f4fac6d80319ad5f80f11e17d21e59f5cff409358a7 Jul 07 16:17:30.293634 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.293598 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:30.293717 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.293668 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:30.293717 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:30.293698 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:31.266866 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:31.266827 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerStarted","Data":"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926"} Jul 07 16:17:31.267270 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:31.266889 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerStarted","Data":"58e36738555dcba00b275f4fac6d80319ad5f80f11e17d21e59f5cff409358a7"} Jul 07 16:17:32.135469 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:32.135436 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:32.141610 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:32.141584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:32.173497 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:32.173445 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podStartSLOduration=3.173428847 podStartE2EDuration="3.173428847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:17:31.307232505 +0000 UTC m=+2024.538298798" watchObservedRunningTime="2026-07-07 16:17:32.173428847 +0000 UTC m=+2025.404495137" Jul 07 16:17:32.271814 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:32.271785 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:32.272851 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:32.272828 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:33.370256 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:33.370210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:33.457623 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:33.457584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:33.539406 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:33.539329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:33.824828 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:33.824792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:34.332421 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:34.332372 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:34.428052 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:34.428020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:37.843396 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:37.843356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:38.342417 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:38.342367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:38.778087 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:38.777989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:39.313331 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:39.313140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:41.461274 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:41.461234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:41.552614 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:41.552578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:41.714941 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:41.714843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:41.901750 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:41.901719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:42.312567 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:42.312530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:42.405698 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:42.405643 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:45.938608 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:45.938572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:48.313596 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:48.313541 2562 patch_prober.go:28] interesting pod/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 16:17:48.314004 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:48.313619 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 16:17:49.383577 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.383537 2562 generic.go:356] "Generic (PLEG): container finished" podID="8bb34c60-bac2-446b-8294-1251390811b8" containerID="0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926" exitCode=0 Jul 07 16:17:49.384010 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.383593 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerDied","Data":"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926"} Jul 07 16:17:49.384054 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.384008 2562 scope.go:117] "RemoveContainer" containerID="0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926" Jul 07 16:17:49.385088 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.385054 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:49.385197 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.385142 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:49.385197 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:49.385190 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:17:50.390753 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:50.390717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerStarted","Data":"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97"} Jul 07 16:17:51.136228 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:51.136190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:51.136688 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:51.136562 2562 patch_prober.go:28] interesting pod/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 16:17:51.136876 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:51.136851 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 16:17:52.135911 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:52.135863 2562 patch_prober.go:28] interesting pod/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 16:17:52.136472 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:52.135947 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 16:17:53.136404 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:53.136345 2562 patch_prober.go:28] interesting pod/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 16:17:53.136926 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:53.136441 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 16:17:53.997871 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:53.997828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:54.136152 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.136115 2562 patch_prober.go:28] interesting pod/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 16:17:54.136342 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.136175 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 16:17:54.382793 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.382761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:54.600786 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.600750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:54.701729 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.701627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:54.898422 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:54.898369 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:17:55.140072 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:55.140042 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:55.140350 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:55.140326 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:17:55.141185 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:17:55.141165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:18:01.829156 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:01.829116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:01.932453 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:01.932425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:02.206367 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:02.206273 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:18:02.206560 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:02.206501 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" containerID="cri-o://4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97" gracePeriod=30 Jul 07 16:18:02.216094 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:02.216064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:03.011700 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:03.011660 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:03.316086 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:03.316053 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:03.671296 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:03.671191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:03.768288 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:03.768256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:04.089166 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:04.089125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:07.452862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.452838 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:18:07.481489 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.481401 2562 generic.go:356] "Generic (PLEG): container finished" podID="8bb34c60-bac2-446b-8294-1251390811b8" containerID="4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97" exitCode=0 Jul 07 16:18:07.481489 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.481428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerDied","Data":"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97"} Jul 07 16:18:07.481489 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.481460 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" Jul 07 16:18:07.481807 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.481497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6" event={"ID":"8bb34c60-bac2-446b-8294-1251390811b8","Type":"ContainerDied","Data":"58e36738555dcba00b275f4fac6d80319ad5f80f11e17d21e59f5cff409358a7"} Jul 07 16:18:07.481807 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.481530 2562 scope.go:117] "RemoveContainer" containerID="4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97" Jul 07 16:18:07.493549 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.493350 2562 scope.go:117] "RemoveContainer" containerID="0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926" Jul 07 16:18:07.502271 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.502251 2562 scope.go:117] "RemoveContainer" containerID="4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97" Jul 07 16:18:07.502571 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:18:07.502551 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97\": container with ID starting with 4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97 not found: ID does not exist" containerID="4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97" Jul 07 16:18:07.502626 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.502581 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97"} err="failed to get container status \"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97\": rpc error: code = NotFound desc = could not find container \"4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97\": container with ID starting with 4366e8cf531c56d5cc6abea392b140ff7ca90e4e075b708102743b4576496c97 not found: ID does not exist" Jul 07 16:18:07.502626 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.502601 2562 scope.go:117] "RemoveContainer" containerID="0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926" Jul 07 16:18:07.502903 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:18:07.502879 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926\": container with ID starting with 0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926 not found: ID does not exist" containerID="0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926" Jul 07 16:18:07.502972 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.502913 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926"} err="failed to get container status \"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926\": rpc error: code = NotFound desc = could not find container \"0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926\": container with ID starting with 0bca8803b6c45732e6fd9e9449c3ebda0552937591c7262be9d891d8eb2d1926 not found: ID does not exist" Jul 07 16:18:07.588130 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588137 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588162 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588190 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqllc\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588223 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588243 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588282 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588335 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588319 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588727 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588346 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs\") pod \"8bb34c60-bac2-446b-8294-1251390811b8\" (UID: \"8bb34c60-bac2-446b-8294-1251390811b8\") " Jul 07 16:18:07.588727 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588599 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:18:07.588727 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588638 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:18:07.588727 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588670 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.588932 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588800 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:18:07.588932 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.588910 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:18:07.589526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.589501 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:18:07.590656 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.590634 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:18:07.590736 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.590676 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:18:07.590864 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.590835 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc" (OuterVolumeSpecName: "kube-api-access-cqllc") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "kube-api-access-cqllc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:18:07.590984 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.590956 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bb34c60-bac2-446b-8294-1251390811b8" (UID: "8bb34c60-bac2-446b-8294-1251390811b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:18:07.689414 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689367 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689414 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689406 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bb34c60-bac2-446b-8294-1251390811b8-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689414 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689415 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689414 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689423 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqllc\" (UniqueName: \"kubernetes.io/projected/8bb34c60-bac2-446b-8294-1251390811b8-kube-api-access-cqllc\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689695 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689432 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689695 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689442 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bb34c60-bac2-446b-8294-1251390811b8-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689695 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689451 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.689695 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.689460 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bb34c60-bac2-446b-8294-1251390811b8-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:18:07.808196 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.808153 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:18:07.812689 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.812658 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1dcebd-openshift-default-66fff89785-6zds6"] Jul 07 16:18:07.926785 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:07.926744 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb34c60-bac2-446b-8294-1251390811b8" path="/var/lib/kubelet/pods/8bb34c60-bac2-446b-8294-1251390811b8/volumes" Jul 07 16:18:09.515218 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:09.515124 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:09.616153 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:09.616111 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:09.812310 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:09.812270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:10.362373 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.362339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:10.686939 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.686903 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:18:10.687446 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687429 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:18:10.687526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687447 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ff808c-9391-4fa1-9606-ead4078651dd" containerName="istio-proxy" Jul 07 16:18:10.687526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687478 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.687526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687487 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.687526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687503 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.687526 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687512 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.687703 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687586 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.687703 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.687601 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb34c60-bac2-446b-8294-1251390811b8" containerName="istio-proxy" Jul 07 16:18:10.692291 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.692268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.695098 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.695074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-94a996-openshift-default-dockercfg-zx6qm\"" Jul 07 16:18:10.704246 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.704211 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:18:10.717302 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.717260 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:10.816825 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817027 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817027 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817027 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pw5m\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817027 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817027 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.816991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817277 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.817034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817277 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.817061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817277 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.817087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.817893 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.817870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:10.918063 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.917969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918063 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918063 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pw5m\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918063 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918448 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918448 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918448 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918577 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918577 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918474 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918665 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918826 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918953 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918795 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.918953 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.919041 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.918981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.920776 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.920749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.920776 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.920763 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.927304 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.927273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:10.927479 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:10.927332 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pw5m\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m\") pod \"e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:11.004200 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.004163 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:11.155681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.155655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:18:11.157756 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:18:11.157724 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8ef432_6c87_481b_96ab_47cdbb34bd1d.slice/crio-69139fff60cca73a2e4a4a2d64089353383a589e327fc01a8e5b7c5e6693d7a4 WatchSource:0}: Error finding container 69139fff60cca73a2e4a4a2d64089353383a589e327fc01a8e5b7c5e6693d7a4: Status 404 returned error can't find the container with id 69139fff60cca73a2e4a4a2d64089353383a589e327fc01a8e5b7c5e6693d7a4 Jul 07 16:18:11.159843 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.159805 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:11.159936 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.159875 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:11.159936 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.159908 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:11.506683 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.506577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerStarted","Data":"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204"} Jul 07 16:18:11.506683 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.506647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerStarted","Data":"69139fff60cca73a2e4a4a2d64089353383a589e327fc01a8e5b7c5e6693d7a4"} Jul 07 16:18:11.532970 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:11.532918 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podStartSLOduration=1.532904029 podStartE2EDuration="1.532904029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:11.53058879 +0000 UTC m=+2064.761655065" watchObservedRunningTime="2026-07-07 16:18:11.532904029 +0000 UTC m=+2064.763970303" Jul 07 16:18:12.004931 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.004894 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:12.010525 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.010489 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:12.511709 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.511679 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:12.512686 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.512667 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:12.691990 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.691944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:12.790637 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:12.790601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:13.532882 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:13.532835 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:13.644653 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:13.644618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:13.766025 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:13.765973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:14.201002 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:14.200966 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:14.387620 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:14.387587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:14.504206 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:14.504094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:18.004458 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:18.004356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:18.318652 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:18.318622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:18.710708 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:18.710603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:18.878835 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:18.878797 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:21.675912 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:21.675876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:21.761358 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:21.761315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:21.856156 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:21.856121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:22.082820 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:22.082786 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:22.475519 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:22.475421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:22.576410 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:22.576350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:26.144665 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.144628 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:26.591137 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.591057 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerID="464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204" exitCode=0 Jul 07 16:18:26.591137 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.591125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerDied","Data":"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204"} Jul 07 16:18:26.591540 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.591522 2562 scope.go:117] "RemoveContainer" containerID="464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204" Jul 07 16:18:26.592506 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.592477 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:26.592559 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.592546 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:26.592606 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:26.592577 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 16:18:27.511707 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:27.511664 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:27.603490 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:27.603455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerStarted","Data":"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892"} Jul 07 16:18:28.004602 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:28.004559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:28.005898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:28.005867 2562 patch_prober.go:28] interesting pod/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 07 16:18:28.006049 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:28.005943 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 07 16:18:29.004858 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:29.004819 2562 patch_prober.go:28] interesting pod/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 07 16:18:29.005253 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:29.004883 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 07 16:18:30.004630 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:30.004588 2562 patch_prober.go:28] interesting pod/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 07 16:18:30.004831 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:30.004646 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 07 16:18:31.005209 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:31.005165 2562 patch_prober.go:28] interesting pod/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 07 16:18:31.005659 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:31.005231 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 07 16:18:32.009151 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:32.009103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:32.009615 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:32.009505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:32.010791 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:32.010765 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:18:36.214857 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:36.214814 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:36.314662 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:36.314626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:41.670002 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:41.669963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:43.360788 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:43.360751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:43.466547 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:43.466514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:18:47.515950 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:47.515921 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:18:47.520906 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:47.520880 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:18:47.523587 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:47.523567 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:18:47.528565 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:18:47.528545 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:19:18.820938 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:19:18.820899 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:19:19.467641 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:19:19.467601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:19:30.468171 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:19:30.468132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:19:37.014486 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:19:37.014448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:21:29.467559 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:21:29.467476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:21:30.549501 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:21:30.549467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:22:01.866166 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:22:01.866137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:22:26.524725 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:22:26.524691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:22:33.074368 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:22:33.074335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:09.770148 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:09.770109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:10.749196 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:10.749166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:13.625817 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:13.625783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:13.723941 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:13.723909 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:27.769041 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:27.769002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:37.081453 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:37.081399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:37.189627 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:37.189592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:44.108853 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:44.108819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:44.209020 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:44.208989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:23:47.556194 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:47.556160 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:23:47.566237 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:47.566211 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:23:47.566878 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:47.566858 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:23:47.573269 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:47.573248 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w4qpz_02816b2d-54ac-4243-b607-535be7d41098/ovn-acl-logging/0.log" Jul 07 16:23:52.712751 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:23:52.712718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:03.836760 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:03.836723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:25.301496 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:25.301404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:37.043663 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:37.043602 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:38.036747 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:38.036716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:39.079170 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:39.079127 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:39.445186 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:39.445158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:39.638729 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:39.638693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:39.739110 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:39.739023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:39.912261 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:39.912232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:24:44.523991 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:24:44.523957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:19.492506 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:19.492472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:19.851092 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:19.851057 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:25:19.851312 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:19.851287 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" containerID="cri-o://ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892" gracePeriod=30 Jul 07 16:25:19.892251 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:19.892220 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:20.349689 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:20.349658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:20.702314 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:20.702210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:20.821546 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:20.821515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:20.921183 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:20.921153 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:21.090781 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:21.090752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:25.104081 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.104057 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:25:25.107784 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107760 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.107869 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107803 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.107869 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107829 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.107869 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107852 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108014 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107913 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108014 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107962 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108014 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.107996 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pw5m\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108141 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108031 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108141 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:25.108141 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108075 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy\") pod \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\" (UID: \"1e8ef432-6c87-481b-96ab-47cdbb34bd1d\") " Jul 07 16:25:25.108268 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108212 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:25.108268 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108226 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:25:25.108428 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108412 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istiod-ca-cert\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.108501 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108430 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.108501 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108438 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-credential-socket\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.108633 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108597 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:25.108836 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.108792 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:25.110602 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.110535 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m" (OuterVolumeSpecName: "kube-api-access-9pw5m") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "kube-api-access-9pw5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:25:25.110705 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.110652 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:25:25.110784 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.110759 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:25:25.110872 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.110854 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e8ef432-6c87-481b-96ab-47cdbb34bd1d" (UID: "1e8ef432-6c87-481b-96ab-47cdbb34bd1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:25.208898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208857 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-envoy\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.208898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208889 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-token\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.208898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208897 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-data\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.208898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208905 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-istio-podinfo\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.209174 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208913 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-workload-certs\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.209174 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.208921 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pw5m\" (UniqueName: \"kubernetes.io/projected/1e8ef432-6c87-481b-96ab-47cdbb34bd1d-kube-api-access-9pw5m\") on node \"ip-10-0-141-42.ec2.internal\" DevicePath \"\"" Jul 07 16:25:25.849501 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.849466 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerID="ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892" exitCode=0 Jul 07 16:25:25.849681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.849523 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" Jul 07 16:25:25.849681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.849545 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerDied","Data":"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892"} Jul 07 16:25:25.849681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.849631 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62" event={"ID":"1e8ef432-6c87-481b-96ab-47cdbb34bd1d","Type":"ContainerDied","Data":"69139fff60cca73a2e4a4a2d64089353383a589e327fc01a8e5b7c5e6693d7a4"} Jul 07 16:25:25.849681 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.849653 2562 scope.go:117] "RemoveContainer" containerID="ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892" Jul 07 16:25:25.859598 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.859582 2562 scope.go:117] "RemoveContainer" containerID="464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204" Jul 07 16:25:25.867878 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.867860 2562 scope.go:117] "RemoveContainer" containerID="ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892" Jul 07 16:25:25.868198 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:25:25.868175 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892\": container with ID starting with ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892 not found: ID does not exist" containerID="ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892" Jul 07 16:25:25.868244 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.868220 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892"} err="failed to get container status \"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892\": rpc error: code = NotFound desc = could not find container \"ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892\": container with ID starting with ec2e0cbc8ee0a8307533c282cf99480c37fb0eae6bba10267dc9910df5d82892 not found: ID does not exist" Jul 07 16:25:25.868285 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.868246 2562 scope.go:117] "RemoveContainer" containerID="464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204" Jul 07 16:25:25.868567 ip-10-0-141-42 kubenswrapper[2562]: E0707 16:25:25.868551 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204\": container with ID starting with 464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204 not found: ID does not exist" containerID="464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204" Jul 07 16:25:25.868629 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.868572 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204"} err="failed to get container status \"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204\": rpc error: code = NotFound desc = could not find container \"464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204\": container with ID starting with 464e99ea7653155fcd928e76a8931b17caa2e25a4f828be14411363475fe4204 not found: ID does not exist" Jul 07 16:25:25.877367 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.877340 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:25:25.881950 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.881928 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-94a996-openshift-default-547bd78b59-bmf62"] Jul 07 16:25:25.926054 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:25.926019 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" path="/var/lib/kubelet/pods/1e8ef432-6c87-481b-96ab-47cdbb34bd1d/volumes" Jul 07 16:25:27.204871 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:27.204834 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:27.305022 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:27.304991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:27.496982 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:27.496883 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:33.434032 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:33.434000 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-5sgtx_da7e15c2-1fca-46f1-8d7a-137e57e813b6/copy-manifests/0.log" Jul 07 16:25:33.441972 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:33.441947 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-5sgtx_da7e15c2-1fca-46f1-8d7a-137e57e813b6/manager/0.log" Jul 07 16:25:34.346037 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.346007 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/extract/0.log" Jul 07 16:25:34.356645 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.356618 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/util/0.log" Jul 07 16:25:34.363615 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.363597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/pull/0.log" Jul 07 16:25:34.475961 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.475934 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/extract/0.log" Jul 07 16:25:34.483327 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.483303 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/util/0.log" Jul 07 16:25:34.496284 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.496257 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/pull/0.log" Jul 07 16:25:34.619980 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.619897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/util/0.log" Jul 07 16:25:34.630136 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.630053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/pull/0.log" Jul 07 16:25:34.638324 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.638285 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/extract/0.log" Jul 07 16:25:34.748769 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.748740 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/pull/0.log" Jul 07 16:25:34.758051 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.758026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/extract/0.log" Jul 07 16:25:34.765257 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:34.765236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/util/0.log" Jul 07 16:25:36.038101 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:36.038069 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678x64lp_6ce19fbd-5c0f-46fa-9284-64147fd126fd/istio-proxy/0.log" Jul 07 16:25:36.159463 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:36.159434 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gj42c_56e00134-3b20-4048-afcb-b55bce2690a0/discovery/0.log" Jul 07 16:25:36.647122 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:36.647094 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r9vh4_5a7e40a2-2fcc-481a-84aa-5de5def8eba0/payload-pre-processing/0.log" Jul 07 16:25:36.779390 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:36.779360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mv6k_19930a0b-e995-434a-8bf1-d1040522618f/payload-processing/0.log" Jul 07 16:25:36.889066 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:36.889040 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6c7b58d5-x9h75_ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f/router/0.log" Jul 07 16:25:37.455084 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:37.455055 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jv6cm_4bdea585-c221-4107-8cb5-63128725b789/main/0.log" Jul 07 16:25:37.679711 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:37.679682 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-65lcd_1d2d3d68-c079-4c16-b50e-e8a402544676/main/0.log" Jul 07 16:25:42.250415 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:42.250365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:42.338348 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:42.338316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" limit=1024 Jul 07 16:25:45.359927 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:45.359899 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9mlh8_de821d61-5eb5-47bd-a9ce-24fe4b4871f6/global-pull-secret-syncer/0.log" Jul 07 16:25:45.524843 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:45.524812 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mmwfv_33b8eab1-91fd-49bf-a7a7-4fc0e957e76c/konnectivity-agent/0.log" Jul 07 16:25:45.698306 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:45.698227 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-42.ec2.internal_db9c52c440a1356e3cef0bcdda630a66/haproxy/0.log" Jul 07 16:25:50.429827 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.429798 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/extract/0.log" Jul 07 16:25:50.458982 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.458951 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/util/0.log" Jul 07 16:25:50.522932 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.522893 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cdfs5_7c92dc76-e518-4e68-930e-97de2c0f8c50/pull/0.log" Jul 07 16:25:50.557623 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.557593 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/extract/0.log" Jul 07 16:25:50.586990 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.586964 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/util/0.log" Jul 07 16:25:50.618803 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.618779 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vjfn7_67a00f97-673d-434f-9593-e609aed806ec/pull/0.log" Jul 07 16:25:50.659939 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.659905 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/extract/0.log" Jul 07 16:25:50.687248 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.687167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/util/0.log" Jul 07 16:25:50.721851 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.721822 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d87zc_777ffd8a-0ab9-47ea-b2a3-000f817122f9/pull/0.log" Jul 07 16:25:50.754627 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.754596 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/extract/0.log" Jul 07 16:25:50.795194 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.795163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/util/0.log" Jul 07 16:25:50.831900 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:50.831876 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef187586_c3939077-bd1f-448f-a0f2-8e230c76872c/pull/0.log" Jul 07 16:25:53.618149 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:53.618115 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-q75ps_b2b5f3bd-1764-4092-a798-cf68777254c4/cluster-monitoring-operator/0.log" Jul 07 16:25:53.908862 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:53.908741 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srf2g_6b8ce45a-0231-4117-b498-559ea6fef53a/node-exporter/0.log" Jul 07 16:25:53.934764 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:53.934732 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srf2g_6b8ce45a-0231-4117-b498-559ea6fef53a/kube-rbac-proxy/0.log" Jul 07 16:25:53.958155 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:53.958129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srf2g_6b8ce45a-0231-4117-b498-559ea6fef53a/init-textfile/0.log" Jul 07 16:25:54.555307 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:54.555278 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-h7c76_3b5380cb-269b-40b1-8868-e1507d5bc2f3/prometheus-operator-admission-webhook/0.log" Jul 07 16:25:55.394882 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.394841 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm"] Jul 07 16:25:55.395283 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.395242 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" Jul 07 16:25:55.395283 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.395255 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" Jul 07 16:25:55.395364 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.395350 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8ef432-6c87-481b-96ab-47cdbb34bd1d" containerName="istio-proxy" Jul 07 16:25:55.398390 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.398360 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.402296 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.402273 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9dw8\"/\"kube-root-ca.crt\"" Jul 07 16:25:55.403587 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.403567 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9dw8\"/\"openshift-service-ca.crt\"" Jul 07 16:25:55.403690 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.403606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s9dw8\"/\"default-dockercfg-9fs8h\"" Jul 07 16:25:55.413813 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.413787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm"] Jul 07 16:25:55.481615 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.481577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-sys\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.481800 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.481628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-podres\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.481800 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.481698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-proc\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.481800 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.481755 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qksj5\" (UniqueName: \"kubernetes.io/projected/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-kube-api-access-qksj5\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.481800 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.481788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-lib-modules\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582447 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-sys\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582447 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-podres\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-proc\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582512 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qksj5\" (UniqueName: \"kubernetes.io/projected/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-kube-api-access-qksj5\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-lib-modules\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-sys\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582612 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-proc\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582706 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582650 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-podres\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.582898 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.582741 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-lib-modules\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.591764 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.591737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qksj5\" (UniqueName: \"kubernetes.io/projected/a7d8c177-3a94-4082-83b4-b7ee9fffbc8c-kube-api-access-qksj5\") pod \"perf-node-gather-daemonset-xlgxm\" (UID: \"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c\") " pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.709446 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.709333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:55.848284 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:55.848259 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm"] Jul 07 16:25:55.849967 ip-10-0-141-42 kubenswrapper[2562]: W0707 16:25:55.849940 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7d8c177_3a94_4082_83b4_b7ee9fffbc8c.slice/crio-00bebf13d9fe874572a0ddcd2db35ae1d09599a94b0c12f3acc29ffd440a740a WatchSource:0}: Error finding container 00bebf13d9fe874572a0ddcd2db35ae1d09599a94b0c12f3acc29ffd440a740a: Status 404 returned error can't find the container with id 00bebf13d9fe874572a0ddcd2db35ae1d09599a94b0c12f3acc29ffd440a740a Jul 07 16:25:56.005458 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.005344 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" event={"ID":"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c","Type":"ContainerStarted","Data":"780c390c8ba8012ea6f67026346663792ac021c06d5018c0bcfe38e27b4e6ed3"} Jul 07 16:25:56.005641 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.005514 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:25:56.005641 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.005544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" event={"ID":"a7d8c177-3a94-4082-83b4-b7ee9fffbc8c","Type":"ContainerStarted","Data":"00bebf13d9fe874572a0ddcd2db35ae1d09599a94b0c12f3acc29ffd440a740a"} Jul 07 16:25:56.026493 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.026438 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" podStartSLOduration=1.026424245 podStartE2EDuration="1.026424245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:56.024356395 +0000 UTC m=+2529.255422671" watchObservedRunningTime="2026-07-07 16:25:56.026424245 +0000 UTC m=+2529.257490521" Jul 07 16:25:56.621340 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.621305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/2.log" Jul 07 16:25:56.625499 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:56.625473 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d96q5_f8286691-e614-41cc-bffe-79ed7194bfab/console-operator/3.log" Jul 07 16:25:57.148558 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:57.148523 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b9d9c5675-k9k9z_e00a76f6-fb8c-4be9-9d4f-5691b7e81f96/console/0.log" Jul 07 16:25:57.185544 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:57.185514 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-c75df_6cf82bc3-6c92-481c-99b1-0ea546d0c5cb/download-server/0.log" Jul 07 16:25:57.806534 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:57.806503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-xdq24_3670b58c-91c7-48c2-98c3-d3dc3ccd010e/volume-data-source-validator/0.log" Jul 07 16:25:59.050737 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:59.050696 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2kbdn_401d0f3f-db12-451d-8452-d3348ccd8de9/dns/0.log" Jul 07 16:25:59.082692 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:59.082662 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2kbdn_401d0f3f-db12-451d-8452-d3348ccd8de9/kube-rbac-proxy/0.log" Jul 07 16:25:59.292471 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:25:59.292446 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sphrk_6be354b0-ab22-49c7-826b-60a7886c2f21/dns-node-resolver/0.log" Jul 07 16:26:00.014963 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:00.014933 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8q4v_9b02124c-6de0-4c0e-83ed-914d2142429c/node-ca/0.log" Jul 07 16:26:01.082742 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:01.082713 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678x64lp_6ce19fbd-5c0f-46fa-9284-64147fd126fd/istio-proxy/0.log" Jul 07 16:26:01.215856 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:01.215817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gj42c_56e00134-3b20-4048-afcb-b55bce2690a0/discovery/0.log" Jul 07 16:26:01.395529 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:01.395441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r9vh4_5a7e40a2-2fcc-481a-84aa-5de5def8eba0/payload-pre-processing/0.log" Jul 07 16:26:01.454130 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:01.454099 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mv6k_19930a0b-e995-434a-8bf1-d1040522618f/payload-processing/0.log" Jul 07 16:26:01.480598 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:01.480561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6c7b58d5-x9h75_ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f/router/0.log" Jul 07 16:26:02.020253 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.020221 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s9dw8/perf-node-gather-daemonset-xlgxm" Jul 07 16:26:02.249250 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.249209 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5rmh7_24bbd909-d7bd-453e-83eb-6c1d5120b132/serve-healthcheck-canary/0.log" Jul 07 16:26:02.855283 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.855244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-bvhfr_f702084c-90f6-49db-a2fd-5a5bee9e76be/insights-operator/0.log" Jul 07 16:26:02.856012 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.855992 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-bvhfr_f702084c-90f6-49db-a2fd-5a5bee9e76be/insights-operator/1.log" Jul 07 16:26:02.886246 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.886217 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8b6q9_034ee060-6915-4f0e-a3e5-904d28a71df2/kube-rbac-proxy/0.log" Jul 07 16:26:02.912988 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.912958 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8b6q9_034ee060-6915-4f0e-a3e5-904d28a71df2/exporter/0.log" Jul 07 16:26:02.941098 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:02.941068 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8b6q9_034ee060-6915-4f0e-a3e5-904d28a71df2/extractor/0.log" Jul 07 16:26:06.146425 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.146374 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-5sgtx_da7e15c2-1fca-46f1-8d7a-137e57e813b6/manager/0.log" Jul 07 16:26:06.174051 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.174020 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f57b78d4-5sgtx_da7e15c2-1fca-46f1-8d7a-137e57e813b6/copy-manifests/0.log" Jul 07 16:26:06.472251 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.472220 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678x64lp_6ce19fbd-5c0f-46fa-9284-64147fd126fd/istio-proxy/0.log" Jul 07 16:26:06.578061 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.578028 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gj42c_56e00134-3b20-4048-afcb-b55bce2690a0/discovery/0.log" Jul 07 16:26:06.703261 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.703171 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r9vh4_5a7e40a2-2fcc-481a-84aa-5de5def8eba0/payload-pre-processing/0.log" Jul 07 16:26:06.745703 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.745674 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mv6k_19930a0b-e995-434a-8bf1-d1040522618f/payload-processing/0.log" Jul 07 16:26:06.764449 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:06.764424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6c7b58d5-x9h75_ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f/router/0.log" Jul 07 16:26:07.642984 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:07.642956 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-646f7cc9db-cfnqt_eb1dbe7a-56cf-4383-9320-09595a3f37bb/manager/0.log" Jul 07 16:26:08.002196 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:08.002164 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678x64lp_6ce19fbd-5c0f-46fa-9284-64147fd126fd/istio-proxy/0.log" Jul 07 16:26:08.127915 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:08.127884 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gj42c_56e00134-3b20-4048-afcb-b55bce2690a0/discovery/0.log" Jul 07 16:26:08.272276 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:08.272181 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r9vh4_5a7e40a2-2fcc-481a-84aa-5de5def8eba0/payload-pre-processing/0.log" Jul 07 16:26:08.313276 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:08.313221 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mv6k_19930a0b-e995-434a-8bf1-d1040522618f/payload-processing/0.log" Jul 07 16:26:08.331026 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:08.330995 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6c7b58d5-x9h75_ca9fda8e-c1fc-41b9-831d-e5ec43d8e94f/router/0.log" Jul 07 16:26:13.614304 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:13.614267 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q8xlf_cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648/kube-storage-version-migrator-operator/1.log" Jul 07 16:26:13.615106 ip-10-0-141-42 kubenswrapper[2562]: I0707 16:26:13.615089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q8xlf_cbbb5a7e-dcbf-4fb5-bf93-85c9fffa8648/kube-storage-version-migrator-operator/0.log"