Jul 08 04:17:33.701583 ip-10-0-142-181 systemd[1]: Starting Kubernetes Kubelet... Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.179050 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186439 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186460 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186464 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186470 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186475 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186478 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186483 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.192596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186486 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186489 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186492 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186495 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186498 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186501 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186505 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186509 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186512 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186515 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186518 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186521 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186524 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186526 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186529 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186531 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186534 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186537 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186540 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186542 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.205092 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186545 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186548 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186561 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186564 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186567 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186570 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186573 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186575 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186578 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186580 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186583 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186586 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186588 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186591 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186593 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186596 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186598 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186601 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186603 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.206014 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186609 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186614 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186617 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186620 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186623 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186626 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186629 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186632 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186635 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186638 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186641 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186644 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186647 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186651 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186653 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186656 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186659 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186662 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186666 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186668 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.207943 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186671 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186673 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186676 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186679 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186683 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186686 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186689 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186692 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186694 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186697 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186700 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186702 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186705 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186707 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186710 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186712 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186715 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186717 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186720 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.209761 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186723 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186725 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186728 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186730 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186733 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186735 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186738 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186740 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186743 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186746 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186749 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186752 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186754 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.186757 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187548 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187557 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187562 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187565 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187568 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187572 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.212603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187575 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187577 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187580 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187582 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187586 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187589 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187591 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187594 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187597 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187600 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187602 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187605 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187610 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187614 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187617 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187620 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187623 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187627 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187629 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.214665 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187632 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187634 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187637 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187641 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187645 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187648 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187651 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187653 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187656 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187659 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187661 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187664 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187667 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187670 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187672 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187674 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187677 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187680 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187683 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187685 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.217461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187688 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187690 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187695 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187698 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187702 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187706 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187709 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187712 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187715 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187718 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187720 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187723 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187726 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187731 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187734 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187737 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187740 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187744 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.218288 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187746 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187749 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187751 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187754 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187756 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187759 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187761 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187763 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187766 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187768 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187771 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187773 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187776 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187779 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187781 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187784 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187786 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187789 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187792 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187795 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.219876 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187798 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187801 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187803 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187806 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187808 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187810 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187813 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187817 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187820 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187822 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187825 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187827 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187831 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187834 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187836 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.187839 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188272 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188280 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188283 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188285 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.221713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188288 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188291 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188294 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188296 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188300 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188304 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188306 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188309 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188312 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188315 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188317 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188320 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188322 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188325 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188329 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188332 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188335 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188338 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188341 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.222568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188346 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188349 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188352 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188355 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188357 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188361 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188363 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188366 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188369 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188372 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188376 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188378 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188381 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188384 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188387 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188389 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188391 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188394 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188396 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188399 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.223568 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188401 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188404 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188406 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188408 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188411 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188414 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188417 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188420 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188422 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188425 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188429 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188432 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188436 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188439 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188442 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188445 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188447 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188450 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188452 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.224596 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188455 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188458 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188460 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188464 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188467 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188469 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188472 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188474 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188477 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188501 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188505 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188508 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188511 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188514 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188517 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188520 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188522 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188525 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188528 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.227057 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188530 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188533 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188535 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188538 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188540 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188543 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188547 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188549 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188552 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188555 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188557 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188560 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188563 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188565 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188568 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188571 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188576 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.188579 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188654 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188663 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 04:17:34.227704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188693 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188698 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188702 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188705 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188710 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188715 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188718 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188721 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188724 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188728 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188731 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188734 2582 flags.go:64] FLAG: --cgroup-root="" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188737 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188740 2582 flags.go:64] FLAG: --client-ca-file="" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188742 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188745 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188750 2582 flags.go:64] FLAG: --cluster-domain="" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188753 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188758 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188761 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188765 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188769 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188772 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 04:17:34.229209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188775 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188778 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188782 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188785 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188788 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188792 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188795 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188799 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188802 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188805 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188808 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188811 2582 flags.go:64] FLAG: --enable-server="true" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188814 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188819 2582 flags.go:64] FLAG: --event-burst="100" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188822 2582 flags.go:64] FLAG: --event-qps="50" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188825 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188828 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188831 2582 flags.go:64] FLAG: --eviction-hard="" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188835 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188838 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188840 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188843 2582 flags.go:64] FLAG: --eviction-soft="" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188846 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188849 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188852 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 04:17:34.229913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188855 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188857 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188862 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188865 2582 flags.go:64] FLAG: --feature-gates="" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188869 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188872 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188875 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188878 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188881 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188884 2582 flags.go:64] FLAG: --help="false" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188887 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188890 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188894 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188897 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188901 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188904 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188907 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188910 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188913 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188916 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188918 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188922 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188924 2582 flags.go:64] FLAG: --kube-reserved="" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188927 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 04:17:34.230606 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188930 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188933 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188936 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188938 2582 flags.go:64] FLAG: --lock-file="" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188941 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188945 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188948 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188953 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188956 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188959 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188963 2582 flags.go:64] FLAG: --logging-format="text" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188966 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188970 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188972 2582 flags.go:64] FLAG: --manifest-url="" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188975 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188980 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188983 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188987 2582 flags.go:64] FLAG: --max-pods="110" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188991 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188994 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.188998 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189001 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189004 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189008 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189010 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 04:17:34.332037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189018 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 04:17:34.271080 ip-10-0-142-181 systemd[1]: Started Kubernetes Kubelet. Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189021 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189024 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189027 2582 flags.go:64] FLAG: --pod-cidr="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189029 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189035 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189038 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189041 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189044 2582 flags.go:64] FLAG: --port="10250" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189047 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189050 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01b8ddbe0bf8c3f13" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189053 2582 flags.go:64] FLAG: --qos-reserved="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189056 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189059 2582 flags.go:64] FLAG: --register-node="true" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189062 2582 flags.go:64] FLAG: --register-with-taints="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189065 2582 flags.go:64] FLAG: --registry-burst="10" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189068 2582 flags.go:64] FLAG: --registry-qps="5" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189071 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189075 2582 flags.go:64] FLAG: --reserved-memory="" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189079 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189082 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189085 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189088 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189091 2582 flags.go:64] FLAG: --runonce="false" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189093 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 04:17:34.333002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189097 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189101 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189105 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189108 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189112 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189115 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189118 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189121 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189124 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189127 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189130 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189133 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189135 2582 flags.go:64] FLAG: --system-cgroups="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189138 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189143 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189146 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189148 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189152 2582 flags.go:64] FLAG: --tls-min-version="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189155 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189158 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189161 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189164 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189166 2582 flags.go:64] FLAG: --v="2" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189170 2582 flags.go:64] FLAG: --version="false" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189174 2582 flags.go:64] FLAG: --vmodule="" Jul 08 04:17:34.334179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189179 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.189183 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189320 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189325 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189328 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189331 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189333 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189336 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189339 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189344 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189346 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189349 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189352 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189354 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189357 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189359 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189362 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189365 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189368 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189370 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.335109 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189373 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189375 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189378 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189380 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189383 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189385 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189388 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189390 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189393 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189396 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189398 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189401 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189405 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189407 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189410 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189413 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189415 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189418 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189422 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.335818 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189425 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189428 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189432 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189435 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189438 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189441 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189444 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189447 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189449 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189452 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189454 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189457 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189459 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189462 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189464 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189467 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189470 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189472 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189474 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189477 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.336668 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189479 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189482 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189484 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189487 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189489 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189493 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189496 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189498 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189502 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189506 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189509 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189511 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189514 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189517 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189521 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189524 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189526 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189529 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189533 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.337644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189537 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189539 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189542 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189544 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189547 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189549 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189552 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189555 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189557 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189560 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189562 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189565 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189567 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189570 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189572 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189575 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189577 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189581 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189585 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189588 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.338643 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189591 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189593 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.189596 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.190417 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.198909 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.198930 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.198952 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.198958 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199017 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199023 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199027 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199030 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199034 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199037 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199040 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.339738 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199043 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199045 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199049 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199053 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199056 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199059 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199063 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199066 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199069 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199071 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199074 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199076 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199079 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199082 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199084 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199087 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199089 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199092 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199095 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199098 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.340592 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199100 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199103 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199107 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199109 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199112 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199114 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199117 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199119 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199122 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199124 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199127 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199129 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199131 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199134 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199136 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199139 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199141 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199144 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199147 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199149 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.341708 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199152 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199154 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199157 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199160 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199163 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199166 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199169 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199171 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199176 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199180 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199182 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199185 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199188 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199190 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199193 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199197 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199199 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199202 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199204 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199207 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.342712 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199210 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199212 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199233 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199235 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199238 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199242 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199246 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199249 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199251 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199254 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199258 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199261 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199264 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199267 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199270 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199272 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199275 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199278 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199281 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.343390 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199283 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199287 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199290 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199292 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199295 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199298 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199301 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199305 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199308 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199312 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199314 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199317 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199319 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.199325 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199448 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:17:34.343985 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199453 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199456 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199459 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199462 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199465 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199475 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199478 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199481 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199484 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199488 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199490 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199493 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199496 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199499 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199502 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199506 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199509 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199512 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199516 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:17:34.344622 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199519 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199521 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199524 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199526 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199529 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199532 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199534 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199536 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199541 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199544 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199546 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199548 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199551 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199553 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199556 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199559 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199561 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199564 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199566 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199569 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:17:34.345461 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199571 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199574 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199578 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199580 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199583 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199586 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199588 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199591 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199593 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199596 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199598 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199603 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199607 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199610 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199613 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199615 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199618 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199621 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199623 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:17:34.346477 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199625 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199628 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199631 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199633 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199636 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199638 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199641 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199643 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199646 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199648 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199651 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199653 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199656 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199659 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199663 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199667 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199670 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199673 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199675 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:17:34.348842 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199678 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199681 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199684 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199686 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199689 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199693 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199696 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199698 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199701 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199704 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199706 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199709 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199711 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199713 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199716 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199719 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199722 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199724 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199727 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199730 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:17:34.349449 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:34.199732 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.199737 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.199979 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.203188 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.208240 2582 server.go:1019] "Starting client certificate rotation" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.208354 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.208398 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.211505 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.217606 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.228300 2582 log.go:25] "Validated CRI v1 runtime API" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.231091 2582 log.go:25] "Validated CRI v1 image API" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.235018 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 04:17:34.459405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.237717 2582 fs.go:135] Filesystem UUIDs: map[179e5120-7a47-4076-98aa-48decf4105df:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 eb73d574-bc54-4bc7-b4b6-28ae955c0e8b:/dev/nvme0n1p4] Jul 08 04:17:34.480947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.237748 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 04:17:34.480947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.240177 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.242870 2582 manager.go:216] Machine: {Timestamp:2026-07-08 04:17:34.242357613 +0000 UTC m=+0.396738747 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099352 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ce9abf2cd3d92ffac33a73d31f69d SystemUUID:ec2ce9ab-f2cd-3d92-ffac-33a73d31f69d BootID:9c8b5f52-0aa2-4d6c-a702-a9dd2639e924 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:24:f6:0f:53:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:24:f6:0f:53:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:fa:ad:0d:04:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.242974 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.243062 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.246734 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.246767 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-181.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.249342 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.249366 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.249375 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.250256 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.252196 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257196 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257355 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257387 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257398 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257410 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257419 2582 kubelet.go:398] "Adding apiserver pod source" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257429 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 04:17:34.481207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257886 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.257904 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.260382 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.263037 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.263065 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265672 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265694 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265703 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265711 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265723 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265731 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265741 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265749 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265759 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265767 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265778 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265787 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265818 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.265826 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.270112 2582 server.go:1268] "Started kubelet" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.270260 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-181.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.270235 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.270329 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.270417 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.270425 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-181.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.270573 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.272689 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.272835 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.275514 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.276382 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-181.ec2.internal.18c0357ade455d2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-181.ec2.internal,UID:ip-10-0-142-181.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-181.ec2.internal,},FirstTimestamp:2026-07-08 04:17:34.270086445 +0000 UTC m=+0.424467560,LastTimestamp:2026-07-08 04:17:34.270086445 +0000 UTC m=+0.424467560,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-181.ec2.internal,}" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.281141 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.281206 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.281245 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282009 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282026 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282273 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282355 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282362 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282840 2582 factory.go:55] Registering systemd factory Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.282855 2582 factory.go:223] Registration of the systemd container factory successfully Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283134 2582 factory.go:153] Registering CRI-O factory Jul 08 04:17:34.481792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283146 2582 factory.go:223] Registration of the crio container factory successfully Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283193 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283212 2582 factory.go:103] Registering Raw factory Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283241 2582 manager.go:1196] Started watching for new ooms in manager Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.283619 2582 manager.go:349] Starting recovery of all containers Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.287412 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-181.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.296925 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qj8kk" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.300541 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.300594 2582 manager.go:354] Recovery completed Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.305004 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.306909 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.306938 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.306951 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.307510 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.307518 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.307520 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.307534 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.308284 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.308313 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.308324 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.308561 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qj8kk" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.309723 2582 policy_none.go:49] "None policy: Start" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.309736 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.309747 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.311738 2582 policy_none.go:47] "Start" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.363010 2582 manager.go:337] "Starting Device Plugin manager" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.363048 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.363057 2582 server.go:87] "Starting device plugin registration server" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.363341 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.363351 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.364504 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.364589 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.364597 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.365163 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.365206 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.385250 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.386521 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.386558 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.386582 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.386614 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.389527 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.463474 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.464690 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.464719 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.464728 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.464776 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.483036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.473421 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.484408 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.473447 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-181.ec2.internal\": node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.487916 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal"] Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.488011 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.489718 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.489760 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.489771 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.492231 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.492381 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.492418 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493445 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493464 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493477 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493485 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493500 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.493488 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.495846 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.495871 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.496709 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.496737 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.496751 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.508210 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.513995 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-181.ec2.internal\" not found" node="ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.519423 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-181.ec2.internal\" not found" node="ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.583974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.584007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.584024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bad711d68fb27849b022a07aaf31df1-config\") pod \"kube-apiserver-proxy-ip-10-0-142-181.ec2.internal\" (UID: \"4bad711d68fb27849b022a07aaf31df1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:34.647619 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:34.608343 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bad711d68fb27849b022a07aaf31df1-config\") pod \"kube-apiserver-proxy-ip-10-0-142-181.ec2.internal\" (UID: \"4bad711d68fb27849b022a07aaf31df1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684549 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684594 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/935776e9b54e889269ed31b5b1736ec5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal\" (UID: \"935776e9b54e889269ed31b5b1736ec5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:34.684617 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4bad711d68fb27849b022a07aaf31df1-config\") pod \"kube-apiserver-proxy-ip-10-0-142-181.ec2.internal\" (UID: \"4bad711d68fb27849b022a07aaf31df1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:35.857453 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:35.761160 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:36.020034 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:35.861965 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:36.020034 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:35.871118 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:36.020034 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:35.877957 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:36.020034 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:35.962802 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:36.166849 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.063286 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:36.166849 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.164014 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-181.ec2.internal\" not found" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.187028 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.234937 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.249644 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.251679 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.256793 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.257004 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.256977 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 04:17:36.279422 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.257061 2582 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://ab54f56ed12144c399be198414139dd5-75af5a5c9c37abb3.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.142.181:45762->32.195.90.187:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.311013 2582 apiserver.go:52] "Watching apiserver" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.327781 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.328159 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-skwn9","kube-system/global-pull-secret-syncer-n7lnz","kube-system/konnectivity-agent-gdsf9","kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz","openshift-image-registry/node-ca-msdnn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal","openshift-multus/multus-additional-cni-plugins-676sm","openshift-network-operator/iptables-alerter-7hbxk","openshift-cluster-node-tuning-operator/tuned-22p2m","openshift-dns/node-resolver-q4m79","openshift-multus/multus-5ljnz","openshift-multus/network-metrics-daemon-dzbws","openshift-network-diagnostics/network-check-target-vjp7n"] Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.330787 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.330874 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.333035 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.333098 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.335079 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.335153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.337368 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.337735 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kgj56\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.337823 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.337850 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.339460 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.339674 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.340354 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.340574 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.341122 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.341251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.341383 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342556 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s2r6g\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342690 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342865 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgft9\" (UniqueName: \"kubernetes.io/projected/b5906284-7602-4e4c-b6ad-e316355875b4-kube-api-access-zgft9\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-kubelet-config\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.380862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342916 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-dbus\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-agent-certs\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.342970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-konnectivity-ca\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.343163 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.343350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.344971 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cmrjf\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.345012 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.345164 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.345276 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.346667 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.346837 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.347977 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.348509 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k254b\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.348748 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.349433 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.350396 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.351233 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.351460 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.351507 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.351716 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.351938 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.352807 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.353204 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wm6b8\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.353472 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.353497 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.353726 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.353748 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fgc29\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.356541 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.357979 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.358328 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.358591 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-twj9h\"" Jul 08 04:17:36.382551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.358639 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.363800 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 04:12:35 +0000 UTC" deadline="2028-03-29 14:56:21.523914784 +0000 UTC" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.363838 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15130h38m45.160079095s" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.364083 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.364107 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.364123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pc7zj\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.364229 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6rfhd\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.364405 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 04:17:36.383874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.383686 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bd9kc" Jul 08 04:17:36.385111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.384377 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bd9kc" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.435835 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443611 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgft9\" (UniqueName: \"kubernetes.io/projected/b5906284-7602-4e4c-b6ad-e316355875b4-kube-api-access-zgft9\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443635 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-env-overrides\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443673 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-systemd\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-run\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.443772 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443788 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-hosts-file\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.443855 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:36.94383186 +0000 UTC m=+2.045670776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443879 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-multus\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443908 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-sys-fs\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-lib-modules\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.443986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-conf-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-node-log\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.444044 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:36.486062 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-log-socket\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.444094 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:36.94408043 +0000 UTC m=+2.045919336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-agent-certs\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-tmp\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444189 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-socket-dir-parent\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-kubelet\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444301 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-ovn\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-netd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444389 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-modprobe-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-tmp-dir\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-netns\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-config\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444476 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 04:17:36.487104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444484 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-socket-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444514 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-registration-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444563 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysconfig\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444577 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-host\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-cni-binary-copy\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-multus-certs\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-slash\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zchsw\" (UniqueName: \"kubernetes.io/projected/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kube-api-access-zchsw\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444691 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fssff\" (UniqueName: \"kubernetes.io/projected/bacccd3a-40f9-4908-9eda-a4f4c930852c-kube-api-access-fssff\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444723 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-var-lib-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-sys\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-var-lib-kubelet\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-hostroot\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-dbus\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-konnectivity-ca\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-k8s-cni-cncf-io\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.488154 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-systemd-units\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-netns\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovn-node-metrics-cert\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444966 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bacccd3a-40f9-4908-9eda-a4f4c930852c-serviceca\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjbcj\" (UniqueName: \"kubernetes.io/projected/db617134-01de-4ba0-97af-ffe6581dfc0e-kube-api-access-zjbcj\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.444999 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-dbus\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-tuned\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l86nj\" (UniqueName: \"kubernetes.io/projected/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-kube-api-access-l86nj\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-script-lib\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdzsv\" (UniqueName: \"kubernetes.io/projected/36a25a04-d10e-4e0b-a9dc-9d4673175941-kube-api-access-wdzsv\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-os-release\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445341 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-bin\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-etc-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445362 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-konnectivity-ca\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.489174 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bacccd3a-40f9-4908-9eda-a4f4c930852c-host\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445427 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-cnibin\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445446 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36a25a04-d10e-4e0b-a9dc-9d4673175941-iptables-alerter-script\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-kubernetes\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-etc-kubernetes\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445520 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-os-release\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36a25a04-d10e-4e0b-a9dc-9d4673175941-host-slash\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-conf\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445576 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-kubelet\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm648\" (UniqueName: \"kubernetes.io/projected/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-kube-api-access-sm648\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-device-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445679 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-system-cni-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-daemon-config\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpqf\" (UniqueName: \"kubernetes.io/projected/04ed614d-84b4-4821-83c8-a1846b5cd752-kube-api-access-nbpqf\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-systemd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445922 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-kubelet-config\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.445972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrqcz\" (UniqueName: \"kubernetes.io/projected/532a49a5-f7ab-41d7-abbc-65a1345e8694-kube-api-access-vrqcz\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446041 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75400726-ae72-406b-b112-64e25c26f5d7-kubelet-config\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446049 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-system-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-cnibin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-bin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.446153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.448149 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811fcce2-8077-4103-8ad3-46a8f6ec7d1d-agent-certs\") pod \"konnectivity-agent-gdsf9\" (UID: \"811fcce2-8077-4103-8ad3-46a8f6ec7d1d\") " pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.490628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.463776 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgft9\" (UniqueName: \"kubernetes.io/projected/b5906284-7602-4e4c-b6ad-e316355875b4-kube-api-access-zgft9\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-modprobe-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-tmp-dir\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-netns\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-config\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546818 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-socket-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546818 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546840 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-registration-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546847 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-netns\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546884 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-modprobe-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546923 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-registration-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546889 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysconfig\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-host\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-cni-binary-copy\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547006 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-socket-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.582500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-multus-certs\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547044 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-multus-certs\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547051 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.546940 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysconfig\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547071 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-host\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-slash\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547099 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-tmp-dir\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zchsw\" (UniqueName: \"kubernetes.io/projected/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kube-api-access-zchsw\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547154 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fssff\" (UniqueName: \"kubernetes.io/projected/bacccd3a-40f9-4908-9eda-a4f4c930852c-kube-api-access-fssff\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547174 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-slash\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547182 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-var-lib-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547198 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-sys\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-var-lib-kubelet\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-hostroot\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-k8s-cni-cncf-io\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-systemd-units\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-netns\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547332 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovn-node-metrics-cert\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.583271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547345 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-var-lib-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547362 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-run-k8s-cni-cncf-io\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547309 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-sys\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547402 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-systemd-units\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bacccd3a-40f9-4908-9eda-a4f4c930852c-serviceca\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547417 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-var-lib-kubelet\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547437 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-hostroot\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjbcj\" (UniqueName: \"kubernetes.io/projected/db617134-01de-4ba0-97af-ffe6581dfc0e-kube-api-access-zjbcj\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-tuned\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l86nj\" (UniqueName: \"kubernetes.io/projected/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-kube-api-access-l86nj\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-script-lib\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdzsv\" (UniqueName: \"kubernetes.io/projected/36a25a04-d10e-4e0b-a9dc-9d4673175941-kube-api-access-wdzsv\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-os-release\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547613 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547639 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-bin\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-etc-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547666 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-cni-binary-copy\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.584256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547777 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bacccd3a-40f9-4908-9eda-a4f4c930852c-serviceca\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547805 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-os-release\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547829 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-netns\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547829 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-bin\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547847 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-etc-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547863 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-openvswitch\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bacccd3a-40f9-4908-9eda-a4f4c930852c-host\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-cnibin\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36a25a04-d10e-4e0b-a9dc-9d4673175941-iptables-alerter-script\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.547996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-kubernetes\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548035 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-etc-kubernetes\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-os-release\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36a25a04-d10e-4e0b-a9dc-9d4673175941-host-slash\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-conf\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548156 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-kubelet\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm648\" (UniqueName: \"kubernetes.io/projected/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-kube-api-access-sm648\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548203 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-kubernetes\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-device-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548181 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-config\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548265 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-etc-kubernetes\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548268 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bacccd3a-40f9-4908-9eda-a4f4c930852c-host\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548307 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-cnibin\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548312 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-os-release\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548356 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-kubelet\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548386 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548390 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovnkube-script-lib\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548434 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36a25a04-d10e-4e0b-a9dc-9d4673175941-host-slash\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548460 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-device-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548575 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-conf\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548620 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-system-cni-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548627 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548667 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36a25a04-d10e-4e0b-a9dc-9d4673175941-iptables-alerter-script\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.585833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548673 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db617134-01de-4ba0-97af-ffe6581dfc0e-system-cni-dir\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-daemon-config\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpqf\" (UniqueName: \"kubernetes.io/projected/04ed614d-84b4-4821-83c8-a1846b5cd752-kube-api-access-nbpqf\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-systemd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548796 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-sysctl-d\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548815 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrqcz\" (UniqueName: \"kubernetes.io/projected/532a49a5-f7ab-41d7-abbc-65a1345e8694-kube-api-access-vrqcz\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-system-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.548986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-cnibin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-bin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-env-overrides\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549137 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-systemd\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-run\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549224 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-daemon-config\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549225 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-systemd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-hosts-file\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-multus\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-sys-fs\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549350 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-lib-modules\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-conf-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-node-log\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549455 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-hosts-file\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549467 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-log-socket\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-tmp\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-socket-dir-parent\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549661 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-kubelet\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-ovn\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-netd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549810 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.586984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549847 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-env-overrides\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549875 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-conf-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549878 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-system-cni-dir\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549906 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-cni-netd\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549925 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-systemd\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549951 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-log-socket\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549959 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-cnibin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549990 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-run\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549994 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-bin\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.549588 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a49a5-f7ab-41d7-abbc-65a1345e8694-lib-modules\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550012 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-node-log\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550058 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-host-run-ovn-kubernetes\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550078 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-multus-socket-dir-parent\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550109 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550119 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-kubelet\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550160 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-run-ovn\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550202 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04ed614d-84b4-4821-83c8-a1846b5cd752-host-var-lib-cni-multus\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550250 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f4c6f09-957c-41b0-9822-aa1d782b32a2-sys-fs\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.587800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550311 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-ovn-node-metrics-cert\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550437 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550456 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-etc-tuned\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.550539 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db617134-01de-4ba0-97af-ffe6581dfc0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.551800 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a49a5-f7ab-41d7-abbc-65a1345e8694-tmp\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.555683 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zchsw\" (UniqueName: \"kubernetes.io/projected/8f4c6f09-957c-41b0-9822-aa1d782b32a2-kube-api-access-zchsw\") pod \"aws-ebs-csi-driver-node-smmwz\" (UID: \"8f4c6f09-957c-41b0-9822-aa1d782b32a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.556564 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fssff\" (UniqueName: \"kubernetes.io/projected/bacccd3a-40f9-4908-9eda-a4f4c930852c-kube-api-access-fssff\") pod \"node-ca-msdnn\" (UID: \"bacccd3a-40f9-4908-9eda-a4f4c930852c\") " pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.556603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l86nj\" (UniqueName: \"kubernetes.io/projected/74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3-kube-api-access-l86nj\") pod \"node-resolver-q4m79\" (UID: \"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3\") " pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.556733 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdzsv\" (UniqueName: \"kubernetes.io/projected/36a25a04-d10e-4e0b-a9dc-9d4673175941-kube-api-access-wdzsv\") pod \"iptables-alerter-7hbxk\" (UID: \"36a25a04-d10e-4e0b-a9dc-9d4673175941\") " pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.556977 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjbcj\" (UniqueName: \"kubernetes.io/projected/db617134-01de-4ba0-97af-ffe6581dfc0e-kube-api-access-zjbcj\") pod \"multus-additional-cni-plugins-676sm\" (UID: \"db617134-01de-4ba0-97af-ffe6581dfc0e\") " pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.558507 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm648\" (UniqueName: \"kubernetes.io/projected/e36ceefe-68ab-49f3-b44e-dcc42c2bbb71-kube-api-access-sm648\") pod \"ovnkube-node-skwn9\" (UID: \"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71\") " pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.558932 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.558947 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.558958 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.559040 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:17:37.05901807 +0000 UTC m=+2.160856991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.562262 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrqcz\" (UniqueName: \"kubernetes.io/projected/532a49a5-f7ab-41d7-abbc-65a1345e8694-kube-api-access-vrqcz\") pod \"tuned-22p2m\" (UID: \"532a49a5-f7ab-41d7-abbc-65a1345e8694\") " pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.567461 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpqf\" (UniqueName: \"kubernetes.io/projected/04ed614d-84b4-4821-83c8-a1846b5cd752-kube-api-access-nbpqf\") pod \"multus-5ljnz\" (UID: \"04ed614d-84b4-4821-83c8-a1846b5cd752\") " pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.588538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.587527 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4m79" Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.595412 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ljnz" Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.625886 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.633268 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935776e9b54e889269ed31b5b1736ec5.slice/crio-c9af4bd6bccf41cf7574ea3c6d614700828b7ad75963d8594c05399a925107bb WatchSource:0}: Error finding container c9af4bd6bccf41cf7574ea3c6d614700828b7ad75963d8594c05399a925107bb: Status 404 returned error can't find the container with id c9af4bd6bccf41cf7574ea3c6d614700828b7ad75963d8594c05399a925107bb Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.635692 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e1a86f_2ba7_47b0_bdf2_fe0f7bd436c3.slice/crio-57f5b21bf6377677257995f7ed2d931373aae88687863e3118b4095edba8b220 WatchSource:0}: Error finding container 57f5b21bf6377677257995f7ed2d931373aae88687863e3118b4095edba8b220: Status 404 returned error can't find the container with id 57f5b21bf6377677257995f7ed2d931373aae88687863e3118b4095edba8b220 Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.635985 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bad711d68fb27849b022a07aaf31df1.slice/crio-4ccd60b090b13b12c98acb0573a8d9ae7667a8aea6fae2bedaabe338ecf5f099 WatchSource:0}: Error finding container 4ccd60b090b13b12c98acb0573a8d9ae7667a8aea6fae2bedaabe338ecf5f099: Status 404 returned error can't find the container with id 4ccd60b090b13b12c98acb0573a8d9ae7667a8aea6fae2bedaabe338ecf5f099 Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.637044 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ed614d_84b4_4821_83c8_a1846b5cd752.slice/crio-c289e0fe918452020780cbbcad9aa2d912cb08e8a6abce174777519f5ed50ae8 WatchSource:0}: Error finding container c289e0fe918452020780cbbcad9aa2d912cb08e8a6abce174777519f5ed50ae8: Status 404 returned error can't find the container with id c289e0fe918452020780cbbcad9aa2d912cb08e8a6abce174777519f5ed50ae8 Jul 08 04:17:36.650500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.639018 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.664527 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.675184 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811fcce2_8077_4103_8ad3_46a8f6ec7d1d.slice/crio-ade28eeac1cc83946035e7bd28645b03ddd55b472e8b82634c07a5a0f0e11790 WatchSource:0}: Error finding container ade28eeac1cc83946035e7bd28645b03ddd55b472e8b82634c07a5a0f0e11790: Status 404 returned error can't find the container with id ade28eeac1cc83946035e7bd28645b03ddd55b472e8b82634c07a5a0f0e11790 Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.695470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.703860 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36ceefe_68ab_49f3_b44e_dcc42c2bbb71.slice/crio-c9373c8e7f9b66afd59df59f802d7752474c1b355494f2d778b69b1103d6417b WatchSource:0}: Error finding container c9373c8e7f9b66afd59df59f802d7752474c1b355494f2d778b69b1103d6417b: Status 404 returned error can't find the container with id c9373c8e7f9b66afd59df59f802d7752474c1b355494f2d778b69b1103d6417b Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.722214 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msdnn" Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.724917 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.732177 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacccd3a_40f9_4908_9eda_a4f4c930852c.slice/crio-6b328ce13d76f8c4ff1b2a2fbac1f7f386bda8a823bc9a42fc6c6502d748e3a2 WatchSource:0}: Error finding container 6b328ce13d76f8c4ff1b2a2fbac1f7f386bda8a823bc9a42fc6c6502d748e3a2: Status 404 returned error can't find the container with id 6b328ce13d76f8c4ff1b2a2fbac1f7f386bda8a823bc9a42fc6c6502d748e3a2 Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.734188 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4c6f09_957c_41b0_9822_aa1d782b32a2.slice/crio-c1e6b13631760699723deeba23e4a66de4f8d0d9e401001ef8fad6cc536756b3 WatchSource:0}: Error finding container c1e6b13631760699723deeba23e4a66de4f8d0d9e401001ef8fad6cc536756b3: Status 404 returned error can't find the container with id c1e6b13631760699723deeba23e4a66de4f8d0d9e401001ef8fad6cc536756b3 Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.738638 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-676sm" Jul 08 04:17:36.778332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.754553 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb617134_01de_4ba0_97af_ffe6581dfc0e.slice/crio-7a42e10de2cc57f074e198f6d0ca560c64b1f0b07f01d81e9576a5e9e81fac59 WatchSource:0}: Error finding container 7a42e10de2cc57f074e198f6d0ca560c64b1f0b07f01d81e9576a5e9e81fac59: Status 404 returned error can't find the container with id 7a42e10de2cc57f074e198f6d0ca560c64b1f0b07f01d81e9576a5e9e81fac59 Jul 08 04:17:36.940683 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.794947 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7hbxk" Jul 08 04:17:36.940683 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.804397 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a25a04_d10e_4e0b_a9dc_9d4673175941.slice/crio-1acc656c93b65217690d9abf7e4df89eaf4c1a495b5d2c9924fa68a145709234 WatchSource:0}: Error finding container 1acc656c93b65217690d9abf7e4df89eaf4c1a495b5d2c9924fa68a145709234: Status 404 returned error can't find the container with id 1acc656c93b65217690d9abf7e4df89eaf4c1a495b5d2c9924fa68a145709234 Jul 08 04:17:36.940683 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.837155 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22p2m" Jul 08 04:17:36.940683 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:17:36.845958 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532a49a5_f7ab_41d7_abbc_65a1345e8694.slice/crio-302855bd4109f119cb335a0d1590428e786d7dccce9e2fe08ed9c7fb76cbc848 WatchSource:0}: Error finding container 302855bd4109f119cb335a0d1590428e786d7dccce9e2fe08ed9c7fb76cbc848: Status 404 returned error can't find the container with id 302855bd4109f119cb335a0d1590428e786d7dccce9e2fe08ed9c7fb76cbc848 Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.955212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:36.955249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.955360 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.955363 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.955409 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:37.955397371 +0000 UTC m=+3.057236277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:37.114654 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:36.955425 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:37.955418583 +0000 UTC m=+3.057257489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:37.227971 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.157638 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:37.227971 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:37.157825 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:37.227971 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:37.157858 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:37.227971 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:37.157871 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:37.227971 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:37.157923 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:17:38.157909635 +0000 UTC m=+3.259748544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:37.399176 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.399106 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 04:12:36 +0000 UTC" deadline="2028-02-25 09:42:27.380587988 +0000 UTC" Jul 08 04:17:37.399176 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.399171 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14333h24m49.981420925s" Jul 08 04:17:37.427613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.427577 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:37.448402 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.448295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4m79" event={"ID":"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3","Type":"ContainerStarted","Data":"57f5b21bf6377677257995f7ed2d931373aae88687863e3118b4095edba8b220"} Jul 08 04:17:37.452663 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.452626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22p2m" event={"ID":"532a49a5-f7ab-41d7-abbc-65a1345e8694","Type":"ContainerStarted","Data":"302855bd4109f119cb335a0d1590428e786d7dccce9e2fe08ed9c7fb76cbc848"} Jul 08 04:17:37.462108 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.461994 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerStarted","Data":"7a42e10de2cc57f074e198f6d0ca560c64b1f0b07f01d81e9576a5e9e81fac59"} Jul 08 04:17:37.467736 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.467699 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msdnn" event={"ID":"bacccd3a-40f9-4908-9eda-a4f4c930852c","Type":"ContainerStarted","Data":"6b328ce13d76f8c4ff1b2a2fbac1f7f386bda8a823bc9a42fc6c6502d748e3a2"} Jul 08 04:17:37.475616 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.475565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gdsf9" event={"ID":"811fcce2-8077-4103-8ad3-46a8f6ec7d1d","Type":"ContainerStarted","Data":"ade28eeac1cc83946035e7bd28645b03ddd55b472e8b82634c07a5a0f0e11790"} Jul 08 04:17:37.478421 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.478338 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ljnz" event={"ID":"04ed614d-84b4-4821-83c8-a1846b5cd752","Type":"ContainerStarted","Data":"c289e0fe918452020780cbbcad9aa2d912cb08e8a6abce174777519f5ed50ae8"} Jul 08 04:17:37.479852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.479821 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" event={"ID":"4bad711d68fb27849b022a07aaf31df1","Type":"ContainerStarted","Data":"4ccd60b090b13b12c98acb0573a8d9ae7667a8aea6fae2bedaabe338ecf5f099"} Jul 08 04:17:37.481727 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.481694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7hbxk" event={"ID":"36a25a04-d10e-4e0b-a9dc-9d4673175941","Type":"ContainerStarted","Data":"1acc656c93b65217690d9abf7e4df89eaf4c1a495b5d2c9924fa68a145709234"} Jul 08 04:17:37.485052 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.485023 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" event={"ID":"8f4c6f09-957c-41b0-9822-aa1d782b32a2","Type":"ContainerStarted","Data":"c1e6b13631760699723deeba23e4a66de4f8d0d9e401001ef8fad6cc536756b3"} Jul 08 04:17:37.488914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.488871 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"c9373c8e7f9b66afd59df59f802d7752474c1b355494f2d778b69b1103d6417b"} Jul 08 04:17:37.492035 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:37.491957 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" event={"ID":"935776e9b54e889269ed31b5b1736ec5","Type":"ContainerStarted","Data":"c9af4bd6bccf41cf7574ea3c6d614700828b7ad75963d8594c05399a925107bb"} Jul 08 04:17:38.007531 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.007491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:38.007712 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.007552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:38.007712 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.007702 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:38.007849 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.007780 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:40.007748991 +0000 UTC m=+5.109587903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:38.008226 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.008205 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:38.008324 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.008265 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:40.008247845 +0000 UTC m=+5.110086757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:38.209346 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.209305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:38.209585 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.209567 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:38.209650 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.209591 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:38.209650 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.209605 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:38.209787 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.209677 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:17:40.209656854 +0000 UTC m=+5.311495763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:38.400418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.400314 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 04:12:36 +0000 UTC" deadline="2028-03-08 07:29:20.496693276 +0000 UTC" Jul 08 04:17:38.400418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.400357 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14619h11m42.096341257s" Jul 08 04:17:38.440389 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.440354 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:38.440578 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.440524 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:38.441020 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.440998 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:38.441161 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.441111 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:38.441231 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.441190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:38.441282 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:38.441268 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:38.538732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:38.538481 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:17:40.029749 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.029702 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:40.030381 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.029827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:40.030381 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.029953 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:40.030381 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.030035 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:44.030012346 +0000 UTC m=+9.131851255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:40.030381 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.030143 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:40.030381 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.030196 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:44.030180927 +0000 UTC m=+9.132019839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:40.231875 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.231789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:40.232078 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.232056 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:40.232160 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.232087 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:40.232160 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.232140 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:40.232269 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.232202 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:17:44.232187759 +0000 UTC m=+9.334026665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:40.440286 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.440248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:40.440486 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.440297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:40.440486 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:40.440258 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:40.440486 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.440400 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:40.440665 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.440516 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:40.440665 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:40.440583 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:42.439404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:42.439374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:42.439917 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:42.439374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:42.439917 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:42.439508 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:42.439917 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:42.439588 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:42.439917 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:42.439385 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:42.439917 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:42.439697 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:44.061827 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.061690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:44.061827 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.061745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:44.062376 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.061883 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:44.062376 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.061895 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:44.062376 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.061956 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:52.061935921 +0000 UTC m=+17.163774827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:44.062376 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.061978 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:17:52.061967841 +0000 UTC m=+17.163806753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:44.264251 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.263675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:44.264251 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.263858 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:44.264251 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.263880 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:44.264251 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.263893 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:44.264251 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.263954 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:17:52.263937525 +0000 UTC m=+17.365776437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:44.439460 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.439380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:44.439460 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.439402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:44.439657 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:44.439380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:44.439657 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.439527 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:44.439657 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.439626 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:44.439819 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:44.439713 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:46.439994 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:46.439951 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:46.440455 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:46.439951 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:46.440455 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:46.440105 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:46.440455 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:46.440203 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:46.440455 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:46.439951 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:46.440455 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:46.440302 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:48.439490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:48.439406 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:48.439490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:48.439414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:48.439975 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:48.439406 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:48.439975 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:48.439548 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:48.439975 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:48.439666 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:48.439975 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:48.439770 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:49.523400 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:49.523132 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22p2m" event={"ID":"532a49a5-f7ab-41d7-abbc-65a1345e8694","Type":"ContainerStarted","Data":"0d2d2747fb9084f1c819545d4b4e600102480648e63aae6a770bc2a880e7ba32"} Jul 08 04:17:49.534580 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:49.534500 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" event={"ID":"4bad711d68fb27849b022a07aaf31df1","Type":"ContainerStarted","Data":"de15dc948e7a95302dc4d3799425dd97d2d7e32f0a305e52e413b9be67f00068"} Jul 08 04:17:49.543816 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:49.543734 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-22p2m" podStartSLOduration=2.816655098 podStartE2EDuration="14.543714975s" totalImagesPullingTime="11.727059877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:49.543564641 +0000 UTC m=+14.645403569" watchObservedRunningTime="2026-07-08 04:17:49.543714975 +0000 UTC m=+14.645553906" Jul 08 04:17:49.559280 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:49.559236 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-181.ec2.internal" podStartSLOduration=13.559222173 podStartE2EDuration="13.559222173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:49.559156859 +0000 UTC m=+14.660995787" watchObservedRunningTime="2026-07-08 04:17:49.559222173 +0000 UTC m=+14.661061101" Jul 08 04:17:50.439702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.439541 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:50.439702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.439570 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:50.439702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.439573 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:50.439702 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:50.439680 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:50.440133 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:50.439796 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:50.440133 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:50.439854 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:50.538857 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.538817 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerStarted","Data":"03323379229f0f9c159350221d03ecaa4f7b3d01919e74fe36eef31d6953bb5d"} Jul 08 04:17:50.540913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.540877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msdnn" event={"ID":"bacccd3a-40f9-4908-9eda-a4f4c930852c","Type":"ContainerStarted","Data":"4accb43458a1672276b443b0e12f45e1404f6e18eab6fae37ef52b08e486a0de"} Jul 08 04:17:50.543153 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.543119 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gdsf9" event={"ID":"811fcce2-8077-4103-8ad3-46a8f6ec7d1d","Type":"ContainerStarted","Data":"9d96aa1278f5963fa63ef951d4dc14928a1ac12edd2115263d57b4f22e577b0d"} Jul 08 04:17:50.545167 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.545135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7hbxk" event={"ID":"36a25a04-d10e-4e0b-a9dc-9d4673175941","Type":"ContainerStarted","Data":"5f521b8de742f946f06faff90233bc1bbf4b90ed8fbe013b783af0c78ed2f5b5"} Jul 08 04:17:50.547632 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.547598 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" event={"ID":"8f4c6f09-957c-41b0-9822-aa1d782b32a2","Type":"ContainerStarted","Data":"a17c0a39f64908f6106d59570056579efec9786483003e6bd61e93edcf0df852"} Jul 08 04:17:50.549418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.549384 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" event={"ID":"935776e9b54e889269ed31b5b1736ec5","Type":"ContainerStarted","Data":"b59fcf135336cd8010743b1776d68fcdac31bfb00f42df2fb84ba0611d1c3d79"} Jul 08 04:17:50.551661 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.551268 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4m79" event={"ID":"74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3","Type":"ContainerStarted","Data":"f968e006a2ad35e15cf6a181a0eda7ce9bcc4e061f9e30cc2efcfa199d4080e1"} Jul 08 04:17:50.581913 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.581852 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-msdnn" podStartSLOduration=3.741545664 podStartE2EDuration="15.581831693s" totalImagesPullingTime="11.840286029s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:50.581812965 +0000 UTC m=+15.683651892" watchObservedRunningTime="2026-07-08 04:17:50.581831693 +0000 UTC m=+15.683670625" Jul 08 04:17:50.597962 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.597907 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7hbxk" podStartSLOduration=3.824281117 podStartE2EDuration="15.597885264s" totalImagesPullingTime="11.773604147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:50.597832026 +0000 UTC m=+15.699670974" watchObservedRunningTime="2026-07-08 04:17:50.597885264 +0000 UTC m=+15.699724193" Jul 08 04:17:50.616899 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.616846 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q4m79" podStartSLOduration=3.68196341 podStartE2EDuration="15.616826237s" totalImagesPullingTime="11.934862827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:50.616431617 +0000 UTC m=+15.718270566" watchObservedRunningTime="2026-07-08 04:17:50.616826237 +0000 UTC m=+15.718665164" Jul 08 04:17:50.659404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:50.659354 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gdsf9" podStartSLOduration=3.762076601 podStartE2EDuration="15.659336919s" totalImagesPullingTime="11.897260318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:50.658405042 +0000 UTC m=+15.760243971" watchObservedRunningTime="2026-07-08 04:17:50.659336919 +0000 UTC m=+15.761175862" Jul 08 04:17:51.103300 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:51.103267 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:51.103905 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:51.103880 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:51.554108 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:51.553994 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:51.554600 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:51.554570 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gdsf9" Jul 08 04:17:52.129003 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.128741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:52.129246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.129038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:52.129246 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.128899 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:52.129246 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.129155 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:08.129127125 +0000 UTC m=+33.230966073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:17:52.129246 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.129175 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:52.129246 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.129226 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:08.129210146 +0000 UTC m=+33.231049104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:17:52.330336 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.330305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:52.330525 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.330496 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:17:52.330525 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.330519 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:17:52.330653 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.330532 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:52.330653 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.330607 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:18:08.330588581 +0000 UTC m=+33.432427505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:17:52.440134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.440039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:52.440134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.440066 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:52.440134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:52.440066 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:52.440383 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.440175 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:52.440383 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.440245 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:52.440383 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:52.440322 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:54.439643 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:54.439600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:54.439643 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:54.439630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:54.440286 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:54.439630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:54.440286 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:54.439743 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:54.440286 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:54.439813 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:54.440286 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:54.439898 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:56.440322 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:56.440280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:56.440802 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:56.440280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:56.440802 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:56.440440 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:56.440802 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:56.440280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:56.440802 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:56.440509 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:56.440802 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:56.440590 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:57.567668 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:57.567516 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="03323379229f0f9c159350221d03ecaa4f7b3d01919e74fe36eef31d6953bb5d" exitCode=0 Jul 08 04:17:57.568256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:57.567585 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"03323379229f0f9c159350221d03ecaa4f7b3d01919e74fe36eef31d6953bb5d"} Jul 08 04:17:57.569556 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:57.569533 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ljnz" event={"ID":"04ed614d-84b4-4821-83c8-a1846b5cd752","Type":"ContainerStarted","Data":"58117110f5c35798bbd39bd8d6dc37af1129c0e2e3c29513c08553aa62f130b7"} Jul 08 04:17:57.571333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:57.571311 2582 generic.go:356] "Generic (PLEG): container finished" podID="935776e9b54e889269ed31b5b1736ec5" containerID="b59fcf135336cd8010743b1776d68fcdac31bfb00f42df2fb84ba0611d1c3d79" exitCode=0 Jul 08 04:17:57.571433 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:57.571347 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" event={"ID":"935776e9b54e889269ed31b5b1736ec5","Type":"ContainerDied","Data":"b59fcf135336cd8010743b1776d68fcdac31bfb00f42df2fb84ba0611d1c3d79"} Jul 08 04:17:58.440187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.440095 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:17:58.440187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.440156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:17:58.440479 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.440105 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:17:58.440479 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:58.440238 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:17:58.440479 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:58.440335 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:17:58.440479 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:17:58.440410 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:17:58.577306 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577278 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:17:58.577854 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577635 2582 generic.go:356] "Generic (PLEG): container finished" podID="e36ceefe-68ab-49f3-b44e-dcc42c2bbb71" containerID="6b4accade3f2f9044846892098e347f37d8ba2d88337571bc31e56c37301a32f" exitCode=1 Jul 08 04:17:58.577854 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577713 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"55de05fd4cbbac6910b42a7579b37933e21ef59c23913c3d7f6500ac15dce4d6"} Jul 08 04:17:58.577854 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577823 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"b8dc047acb87e5f0fcd156ec1419dc8252f5fd5a6c3d334bdff651945f499408"} Jul 08 04:17:58.577998 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"e6c96054211f8c01a6aea8846548ee47b51b8fe5ea2238a1a836bb8864611b06"} Jul 08 04:17:58.577998 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"09dc139832614ef1041b914ed714399f09f2dbeade5c7383dfefd143b0872b9e"} Jul 08 04:17:58.577998 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.577976 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerDied","Data":"6b4accade3f2f9044846892098e347f37d8ba2d88337571bc31e56c37301a32f"} Jul 08 04:17:58.578113 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.578069 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"6959f12580ff2c5ca82d4aafb4e6d2e8c7d872584c183a341339edbc3a74ff38"} Jul 08 04:17:58.579852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.579828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" event={"ID":"935776e9b54e889269ed31b5b1736ec5","Type":"ContainerStarted","Data":"56461ee0d2f5fdf6b1637233b432d2f982338a38e17259e45175f63b124a9988"} Jul 08 04:17:58.618064 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.617993 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-181.ec2.internal" podStartSLOduration=22.617969881 podStartE2EDuration="22.617969881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:58.617804896 +0000 UTC m=+23.719643826" watchObservedRunningTime="2026-07-08 04:17:58.617969881 +0000 UTC m=+23.719808811" Jul 08 04:17:58.618491 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.618453 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5ljnz" podStartSLOduration=2.790051104 podStartE2EDuration="23.618439034s" totalImagesPullingTime="20.82838793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:17:58.600457908 +0000 UTC m=+23.702296835" watchObservedRunningTime="2026-07-08 04:17:58.618439034 +0000 UTC m=+23.720277965" Jul 08 04:17:58.767828 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:58.767621 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 04:17:59.431602 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:59.431488 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T04:17:58.767819909Z","UUID":"55446549-5e6d-4821-8dfa-7c8ca4d81337","Handler":null,"Name":"","Endpoint":""} Jul 08 04:17:59.433183 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:59.433159 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 04:17:59.433312 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:59.433198 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 04:17:59.585026 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:17:59.584990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" event={"ID":"8f4c6f09-957c-41b0-9822-aa1d782b32a2","Type":"ContainerStarted","Data":"89233ed347f62b273799e09b99caba810529acb838bb95cf365ff08b11383992"} Jul 08 04:18:00.440228 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.440156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:00.440371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.440255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:00.440371 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:00.440261 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:00.440371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.440285 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:00.440476 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:00.440374 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:00.440476 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:00.440425 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:00.589601 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.589560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" event={"ID":"8f4c6f09-957c-41b0-9822-aa1d782b32a2","Type":"ContainerStarted","Data":"79b8182797a599c2d1ee1ef1acffdb427bf54f2c73ed10c1385a4ddd7adaf2b5"} Jul 08 04:18:00.592725 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.592702 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:18:00.593115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.593092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"28eff0b6757996a531c7c4fb2d88c5d8733da346c7a2e55076ac8dff022fe932"} Jul 08 04:18:00.610449 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:00.610400 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-smmwz" podStartSLOduration=3.9190575279999997 podStartE2EDuration="25.610385036s" totalImagesPullingTime="21.691327508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:00.609854355 +0000 UTC m=+25.711693304" watchObservedRunningTime="2026-07-08 04:18:00.610385036 +0000 UTC m=+25.712223963" Jul 08 04:18:02.440166 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:02.440127 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:02.440166 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:02.440160 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:02.440665 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:02.440144 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:02.440665 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:02.440263 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:02.440665 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:02.440364 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:02.440665 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:02.440473 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:03.603729 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:03.603697 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:18:03.604596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:03.604151 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"3ea3dd89d83e0ad3d07bac42c1b8444f9ff8c131235b4a57db0d1f5c01e5894c"} Jul 08 04:18:03.604596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:03.604504 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:03.604721 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:03.604647 2582 scope.go:117] "RemoveContainer" containerID="6b4accade3f2f9044846892098e347f37d8ba2d88337571bc31e56c37301a32f" Jul 08 04:18:03.626031 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:03.626002 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:04.440260 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.440060 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:04.440481 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.440063 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:04.440481 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:04.440352 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:04.440481 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.440067 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:04.440481 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:04.440431 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:04.440683 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:04.440510 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:04.611435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.611307 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:18:04.613656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.612272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" event={"ID":"e36ceefe-68ab-49f3-b44e-dcc42c2bbb71","Type":"ContainerStarted","Data":"a19ba4ad75a7b7bf78e589f017688919201120b251dcc64b65b5c3cb94a71d79"} Jul 08 04:18:04.613656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.613508 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:04.613656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.613591 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:04.617371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.617344 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="ba090df2ece35fad4bc87fb62e6de5ffe325d5ee469f976ee670e83af81eeab5" exitCode=0 Jul 08 04:18:04.617498 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.617394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"ba090df2ece35fad4bc87fb62e6de5ffe325d5ee469f976ee670e83af81eeab5"} Jul 08 04:18:04.631311 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.631289 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:04.642086 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:04.642040 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" podStartSLOduration=8.857611993 podStartE2EDuration="29.642025394s" totalImagesPullingTime="20.784413401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:04.640734422 +0000 UTC m=+29.742573382" watchObservedRunningTime="2026-07-08 04:18:04.642025394 +0000 UTC m=+29.743864343" Jul 08 04:18:05.336241 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.336207 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7lnz"] Jul 08 04:18:05.336421 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.336405 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:05.336562 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:05.336537 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:05.339209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.339180 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjp7n"] Jul 08 04:18:05.339367 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.339322 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:05.339503 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:05.339476 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:05.339864 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.339844 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dzbws"] Jul 08 04:18:05.339967 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:05.339943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:05.340061 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:05.340044 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:06.623523 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:06.623488 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="c0d1afc8da5acd3c28c51998fd5b6daa1a818440e7893412aa82f3f8035b514a" exitCode=0 Jul 08 04:18:06.623929 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:06.623555 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"c0d1afc8da5acd3c28c51998fd5b6daa1a818440e7893412aa82f3f8035b514a"} Jul 08 04:18:07.439966 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:07.439879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:07.440101 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:07.439879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:07.440101 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:07.440022 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:07.440179 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:07.440129 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:07.440179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:07.439880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:07.440261 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:07.440241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:08.145750 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:08.145501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:08.145750 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:08.145701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:08.146187 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.145657 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:08.146187 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.145819 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:08.146187 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.145831 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs podName:b5906284-7602-4e4c-b6ad-e316355875b4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:40.145815246 +0000 UTC m=+65.247654152 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs") pod "network-metrics-daemon-dzbws" (UID: "b5906284-7602-4e4c-b6ad-e316355875b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:08.146187 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.145856 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret podName:75400726-ae72-406b-b112-64e25c26f5d7 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:40.145846289 +0000 UTC m=+65.247685195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret") pod "global-pull-secret-syncer-n7lnz" (UID: "75400726-ae72-406b-b112-64e25c26f5d7") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:08.347965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:08.347916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:08.348143 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.348086 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:08.348143 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.348109 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:08.348143 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.348122 2582 projected.go:196] Error preparing data for projected volume kube-api-access-lrds5 for pod openshift-network-diagnostics/network-check-target-vjp7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:08.348243 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:08.348179 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5 podName:7117c335-b1a4-4d8a-b896-51919001f08c nodeName:}" failed. No retries permitted until 2026-07-08 04:18:40.348162063 +0000 UTC m=+65.450000970 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrds5" (UniqueName: "kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5") pod "network-check-target-vjp7n" (UID: "7117c335-b1a4-4d8a-b896-51919001f08c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:08.630991 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:08.630958 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="916465897f6087dc69241b35d2a032cb4177146a8421bf0b53fa43479e258c24" exitCode=0 Jul 08 04:18:08.631144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:08.631014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"916465897f6087dc69241b35d2a032cb4177146a8421bf0b53fa43479e258c24"} Jul 08 04:18:09.439749 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:09.439718 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:09.439749 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:09.439719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:09.440229 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:09.439719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:09.440229 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:09.439896 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzbws" podUID="b5906284-7602-4e4c-b6ad-e316355875b4" Jul 08 04:18:09.440229 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:09.440027 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lnz" podUID="75400726-ae72-406b-b112-64e25c26f5d7" Jul 08 04:18:09.440229 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:09.440109 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjp7n" podUID="7117c335-b1a4-4d8a-b896-51919001f08c" Jul 08 04:18:10.214723 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.214633 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-181.ec2.internal" event="NodeReady" Jul 08 04:18:10.214895 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.214822 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 04:18:10.261800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.261744 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr"] Jul 08 04:18:10.265495 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.265474 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" Jul 08 04:18:10.271702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.271239 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.271702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.271586 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.274552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.274525 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:18:10.279880 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.279861 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs"] Jul 08 04:18:10.280792 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.280752 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.282563 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.282336 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wwmc2\"" Jul 08 04:18:10.284646 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.283744 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nl96k"] Jul 08 04:18:10.284646 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.283855 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" Jul 08 04:18:10.287365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.287341 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 04:18:10.287468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.287403 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 04:18:10.287699 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.287682 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 04:18:10.288564 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.288545 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:18:10.288877 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.288630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.293066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.289694 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.293066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.289951 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p4jt\"" Jul 08 04:18:10.293066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.290002 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.293066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.290109 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-f48gv\"" Jul 08 04:18:10.296371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.294489 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj"] Jul 08 04:18:10.296371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.295550 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.296371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.295816 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 04:18:10.296371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.295963 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.296371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.296185 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 04:18:10.297020 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.296880 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.297468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.297449 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bd8nk\"" Jul 08 04:18:10.298229 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.298172 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44"] Jul 08 04:18:10.299025 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.298983 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.302466 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.302446 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx"] Jul 08 04:18:10.302603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.302583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.306077 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.305964 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl"] Jul 08 04:18:10.306808 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.306282 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.311832 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.310929 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 04:18:10.311832 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.311022 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.311832 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.311245 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 04:18:10.311832 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.311398 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 04:18:10.312110 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.311937 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 04:18:10.312167 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.312111 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.312460 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.312441 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 04:18:10.312731 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.310935 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-bdfc5556f-2c48c"] Jul 08 04:18:10.313149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.313130 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.313514 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.313495 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.313911 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.313890 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.314066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.314043 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 04:18:10.314146 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.314130 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.315225 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.315205 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jlbqq\"" Jul 08 04:18:10.315779 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.315745 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.315969 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.315940 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-skfdq\"" Jul 08 04:18:10.316028 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.315965 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 04:18:10.316288 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.316265 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.316587 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.316305 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6pd5b\"" Jul 08 04:18:10.316739 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.316720 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 04:18:10.316932 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.316681 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 04:18:10.317067 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.317052 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.317242 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.317219 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q8xbn\"" Jul 08 04:18:10.317978 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.317961 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-flg5k"] Jul 08 04:18:10.318110 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.318091 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.320591 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.320571 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.320844 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.320826 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.321206 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321028 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d5gcx\"" Jul 08 04:18:10.321206 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321098 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 04:18:10.321206 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321146 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-skrm2"] Jul 08 04:18:10.321401 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321334 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 04:18:10.321401 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321394 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 04:18:10.321543 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321482 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 04:18:10.321588 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.321562 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.326118 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326094 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.326393 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326375 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.326556 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326536 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 04:18:10.326685 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326592 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr"] Jul 08 04:18:10.326866 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326844 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs"] Jul 08 04:18:10.326947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326887 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xcmg"] Jul 08 04:18:10.326947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.326926 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 04:18:10.331196 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.330528 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.331196 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.331132 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-449hv\"" Jul 08 04:18:10.331917 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.331896 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 04:18:10.332024 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.331931 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj"] Jul 08 04:18:10.332024 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.331973 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl"] Jul 08 04:18:10.332024 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.331998 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:18:10.332024 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.332022 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:18:10.332215 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.332040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx"] Jul 08 04:18:10.332215 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.332060 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bdfc5556f-2c48c"] Jul 08 04:18:10.332215 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.332082 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sdfj6"] Jul 08 04:18:10.332215 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.332131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.333218 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.333159 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 04:18:10.333633 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.333616 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8jn4l\"" Jul 08 04:18:10.333967 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.333771 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.335941 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-flg5k"] Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.335993 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nl96k"] Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336024 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-skrm2"] Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336048 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sdfj6"] Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336069 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xcmg"] Jul 08 04:18:10.336175 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336098 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44"] Jul 08 04:18:10.336467 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336195 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.336467 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336237 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xh8sv\"" Jul 08 04:18:10.336467 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336339 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 04:18:10.336668 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.336535 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 04:18:10.339098 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.339074 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 04:18:10.339191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.339150 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 04:18:10.339680 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.339662 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v9th6\"" Jul 08 04:18:10.339785 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.339734 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 04:18:10.363292 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.363418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.363418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363320 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.363418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.363418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363369 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca7f6469-7920-4c58-92be-eb8dc3791202-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.363418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.363593 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.363593 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.363593 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.363593 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsm8c\" (UniqueName: \"kubernetes.io/projected/63781dc0-d73d-4209-9cd7-49951c8693eb-kube-api-access-hsm8c\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.363593 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwn9\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363674 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63781dc0-d73d-4209-9cd7-49951c8693eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63781dc0-d73d-4209-9cd7-49951c8693eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363719 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdb76\" (UniqueName: \"kubernetes.io/projected/98ab92c6-780e-439c-b578-df03697fa3e1-kube-api-access-sdb76\") pod \"network-check-source-989f8b8dc-l5bxr\" (UID: \"98ab92c6-780e-439c-b578-df03697fa3e1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.363851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363824 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nj9q\" (UniqueName: \"kubernetes.io/projected/0997be05-260b-4d34-b609-24723de13d50-kube-api-access-4nj9q\") pod \"volume-data-source-validator-5d456fdbcd-x88qs\" (UID: \"0997be05-260b-4d34-b609-24723de13d50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqltg\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363940 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-tmp\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.363990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbknj\" (UniqueName: \"kubernetes.io/projected/4ecfabab-c5bf-425e-9d4e-a74550dd2754-kube-api-access-gbknj\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364085 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ecfabab-c5bf-425e-9d4e-a74550dd2754-serving-cert\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.364137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.364404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364137 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.364404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.364404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt5dp\" (UniqueName: \"kubernetes.io/projected/ca7f6469-7920-4c58-92be-eb8dc3791202-kube-api-access-wt5dp\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.364404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-snapshots\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.364404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.364322 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.464639 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464606 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.464639 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-default-certificate\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464662 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsm8c\" (UniqueName: \"kubernetes.io/projected/63781dc0-d73d-4209-9cd7-49951c8693eb-kube-api-access-hsm8c\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwn9\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63781dc0-d73d-4209-9cd7-49951c8693eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63781dc0-d73d-4209-9cd7-49951c8693eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.464971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdb76\" (UniqueName: \"kubernetes.io/projected/98ab92c6-780e-439c-b578-df03697fa3e1-kube-api-access-sdb76\") pod \"network-check-source-989f8b8dc-l5bxr\" (UID: \"98ab92c6-780e-439c-b578-df03697fa3e1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f798bae3-92ec-4263-addb-28e02abf35f4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nj9q\" (UniqueName: \"kubernetes.io/projected/0997be05-260b-4d34-b609-24723de13d50-kube-api-access-4nj9q\") pod \"volume-data-source-validator-5d456fdbcd-x88qs\" (UID: \"0997be05-260b-4d34-b609-24723de13d50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" Jul 08 04:18:10.465105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7q25\" (UniqueName: \"kubernetes.io/projected/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-kube-api-access-q7q25\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465118 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk6z7\" (UniqueName: \"kubernetes.io/projected/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-kube-api-access-fk6z7\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465231 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1da944-0214-4cd8-87be-838ee702e3ca-cert\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqltg\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465303 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-tmp\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465331 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10be44bf-2a06-440b-baf5-951568059b8c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465389 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbknj\" (UniqueName: \"kubernetes.io/projected/4ecfabab-c5bf-425e-9d4e-a74550dd2754-kube-api-access-gbknj\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465454 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/63781dc0-d73d-4209-9cd7-49951c8693eb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465471 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ecfabab-c5bf-425e-9d4e-a74550dd2754-serving-cert\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.465573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10be44bf-2a06-440b-baf5-951568059b8c-config\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.465999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-config-volume\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466160 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466193 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466194 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-tmp\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466208 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:10.466248 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466212 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466268 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:10.96624989 +0000 UTC m=+36.068088822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt5dp\" (UniqueName: \"kubernetes.io/projected/ca7f6469-7920-4c58-92be-eb8dc3791202-kube-api-access-wt5dp\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-snapshots\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-serving-cert\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-tmp-dir\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466483 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-stats-auth\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466570 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466596 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-config\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466622 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466642 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:10.466708 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466656 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.466706 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:10.966691704 +0000 UTC m=+36.068530627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466653 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca7f6469-7920-4c58-92be-eb8dc3791202-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466749 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwgkr\" (UniqueName: \"kubernetes.io/projected/10be44bf-2a06-440b-baf5-951568059b8c-kube-api-access-fwgkr\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-trusted-ca\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq4sf\" (UniqueName: \"kubernetes.io/projected/7d1da944-0214-4cd8-87be-838ee702e3ca-kube-api-access-dq4sf\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466905 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ecfabab-c5bf-425e-9d4e-a74550dd2754-snapshots\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjnw7\" (UniqueName: \"kubernetes.io/projected/51e092a6-2a2d-4df1-8be9-81078a8081c1-kube-api-access-pjnw7\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.466971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467057 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c5l4\" (UniqueName: \"kubernetes.io/projected/e0fd4931-ebcc-4948-b5fd-748844c3c538-kube-api-access-6c5l4\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467371 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca7f6469-7920-4c58-92be-eb8dc3791202-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.467492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.468253 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.467486 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:10.468253 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.467525 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:10.967513517 +0000 UTC m=+36.069352424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:10.468253 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467859 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.468253 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467874 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.468253 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.467967 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ecfabab-c5bf-425e-9d4e-a74550dd2754-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.470210 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470184 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.470210 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470198 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.470383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470186 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ecfabab-c5bf-425e-9d4e-a74550dd2754-serving-cert\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.470383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470234 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.470383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470320 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63781dc0-d73d-4209-9cd7-49951c8693eb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.470752 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.470734 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.473624 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.473588 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdb76\" (UniqueName: \"kubernetes.io/projected/98ab92c6-780e-439c-b578-df03697fa3e1-kube-api-access-sdb76\") pod \"network-check-source-989f8b8dc-l5bxr\" (UID: \"98ab92c6-780e-439c-b578-df03697fa3e1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" Jul 08 04:18:10.473624 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.473612 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.473908 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.473889 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsm8c\" (UniqueName: \"kubernetes.io/projected/63781dc0-d73d-4209-9cd7-49951c8693eb-kube-api-access-hsm8c\") pod \"kube-storage-version-migrator-operator-846bf668f7-875fj\" (UID: \"63781dc0-d73d-4209-9cd7-49951c8693eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.480628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.480563 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbknj\" (UniqueName: \"kubernetes.io/projected/4ecfabab-c5bf-425e-9d4e-a74550dd2754-kube-api-access-gbknj\") pod \"insights-operator-5c847dcfc4-nl96k\" (UID: \"4ecfabab-c5bf-425e-9d4e-a74550dd2754\") " pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.480935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.480912 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nj9q\" (UniqueName: \"kubernetes.io/projected/0997be05-260b-4d34-b609-24723de13d50-kube-api-access-4nj9q\") pod \"volume-data-source-validator-5d456fdbcd-x88qs\" (UID: \"0997be05-260b-4d34-b609-24723de13d50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" Jul 08 04:18:10.481096 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.480986 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqltg\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.481096 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.481006 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt5dp\" (UniqueName: \"kubernetes.io/projected/ca7f6469-7920-4c58-92be-eb8dc3791202-kube-api-access-wt5dp\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.482580 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.482554 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwn9\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.482692 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.482675 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.568299 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10be44bf-2a06-440b-baf5-951568059b8c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.568468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.568468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568335 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.568468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568374 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10be44bf-2a06-440b-baf5-951568059b8c-config\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.568468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-config-volume\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.568468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-serving-cert\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.568471 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568500 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-tmp-dir\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-stats-auth\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.568540 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.06851937 +0000 UTC m=+36.170358276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-config\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568619 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwgkr\" (UniqueName: \"kubernetes.io/projected/10be44bf-2a06-440b-baf5-951568059b8c-kube-api-access-fwgkr\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.568713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-trusted-ca\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.569268 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq4sf\" (UniqueName: \"kubernetes.io/projected/7d1da944-0214-4cd8-87be-838ee702e3ca-kube-api-access-dq4sf\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.569268 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.568787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjnw7\" (UniqueName: \"kubernetes.io/projected/51e092a6-2a2d-4df1-8be9-81078a8081c1-kube-api-access-pjnw7\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.569268 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569192 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:10.569268 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569246 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.06923011 +0000 UTC m=+36.171069018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:10.569528 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.569393 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-config-volume\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.569528 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569493 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:10.569631 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569545 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.069529809 +0000 UTC m=+36.171368721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:10.570217 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.569708 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-config\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.570217 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569811 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:10.570217 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.569879 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.069863807 +0000 UTC m=+36.171702741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:10.570217 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570051 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-trusted-ca\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.570497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570284 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-tmp-dir\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.570497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c5l4\" (UniqueName: \"kubernetes.io/projected/e0fd4931-ebcc-4948-b5fd-748844c3c538-kube-api-access-6c5l4\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.570497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.570497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-default-certificate\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.570497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570478 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f798bae3-92ec-4263-addb-28e02abf35f4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.570732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7q25\" (UniqueName: \"kubernetes.io/projected/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-kube-api-access-q7q25\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.570732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk6z7\" (UniqueName: \"kubernetes.io/projected/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-kube-api-access-fk6z7\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.570732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1da944-0214-4cd8-87be-838ee702e3ca-cert\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.570732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.570630 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10be44bf-2a06-440b-baf5-951568059b8c-config\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.571105 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.571070 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.071051973 +0000 UTC m=+36.172890885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:10.571413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.571382 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f798bae3-92ec-4263-addb-28e02abf35f4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:10.571656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.571626 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-stats-auth\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.571899 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.571880 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10be44bf-2a06-440b-baf5-951568059b8c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.573408 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.573374 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-serving-cert\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.573496 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.573425 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1da944-0214-4cd8-87be-838ee702e3ca-cert\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.573613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.573591 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-default-certificate\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.580969 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.580948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" Jul 08 04:18:10.582456 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.582407 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq4sf\" (UniqueName: \"kubernetes.io/projected/7d1da944-0214-4cd8-87be-838ee702e3ca-kube-api-access-dq4sf\") pod \"ingress-canary-sdfj6\" (UID: \"7d1da944-0214-4cd8-87be-838ee702e3ca\") " pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.582802 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.582782 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk6z7\" (UniqueName: \"kubernetes.io/projected/d1affeec-0fb0-48c5-9c3c-3b3f701e12a2-kube-api-access-fk6z7\") pod \"console-operator-68b757fb67-flg5k\" (UID: \"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2\") " pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.583118 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.583100 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwgkr\" (UniqueName: \"kubernetes.io/projected/10be44bf-2a06-440b-baf5-951568059b8c-kube-api-access-fwgkr\") pod \"service-ca-operator-5d46d9ffd4-vrrtx\" (UID: \"10be44bf-2a06-440b-baf5-951568059b8c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.583781 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.583740 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c5l4\" (UniqueName: \"kubernetes.io/projected/e0fd4931-ebcc-4948-b5fd-748844c3c538-kube-api-access-6c5l4\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:10.583904 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.583887 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjnw7\" (UniqueName: \"kubernetes.io/projected/51e092a6-2a2d-4df1-8be9-81078a8081c1-kube-api-access-pjnw7\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:10.584056 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.584040 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7q25\" (UniqueName: \"kubernetes.io/projected/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-kube-api-access-q7q25\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:10.606591 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.606563 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" Jul 08 04:18:10.620583 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.620547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" Jul 08 04:18:10.639875 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.639853 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" Jul 08 04:18:10.655337 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.655280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" Jul 08 04:18:10.688094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.687171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:10.698696 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.694001 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sdfj6" Jul 08 04:18:10.780068 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.779604 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr"] Jul 08 04:18:10.793394 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.793260 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ab92c6_780e_439c_b578_df03697fa3e1.slice/crio-6fe327048809b4ebd5c8b40fd1ef9499a19fe040cca2789a52fc095bf0cf2d5b WatchSource:0}: Error finding container 6fe327048809b4ebd5c8b40fd1ef9499a19fe040cca2789a52fc095bf0cf2d5b: Status 404 returned error can't find the container with id 6fe327048809b4ebd5c8b40fd1ef9499a19fe040cca2789a52fc095bf0cf2d5b Jul 08 04:18:10.803371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.802991 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs"] Jul 08 04:18:10.863708 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.863652 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nl96k"] Jul 08 04:18:10.868202 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.867630 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecfabab_c5bf_425e_9d4e_a74550dd2754.slice/crio-fb6a467aaea69adb99c6893ddd414f271115bacecd10a71b85387ca19707c2c1 WatchSource:0}: Error finding container fb6a467aaea69adb99c6893ddd414f271115bacecd10a71b85387ca19707c2c1: Status 404 returned error can't find the container with id fb6a467aaea69adb99c6893ddd414f271115bacecd10a71b85387ca19707c2c1 Jul 08 04:18:10.890647 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.890571 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj"] Jul 08 04:18:10.896405 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.896375 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63781dc0_d73d_4209_9cd7_49951c8693eb.slice/crio-d0d5d68c877223fd7fb82466fef4d71da9057fadf8f46cf936e7d801ffe4a977 WatchSource:0}: Error finding container d0d5d68c877223fd7fb82466fef4d71da9057fadf8f46cf936e7d801ffe4a977: Status 404 returned error can't find the container with id d0d5d68c877223fd7fb82466fef4d71da9057fadf8f46cf936e7d801ffe4a977 Jul 08 04:18:10.907502 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.907468 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx"] Jul 08 04:18:10.914211 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.914175 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10be44bf_2a06_440b_baf5_951568059b8c.slice/crio-ef104bd5d51222343a412ea7dbda4f2626fe5c44f08a80a633e43d311fff12fa WatchSource:0}: Error finding container ef104bd5d51222343a412ea7dbda4f2626fe5c44f08a80a633e43d311fff12fa: Status 404 returned error can't find the container with id ef104bd5d51222343a412ea7dbda4f2626fe5c44f08a80a633e43d311fff12fa Jul 08 04:18:10.930480 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.930420 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sdfj6"] Jul 08 04:18:10.933419 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.933390 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1da944_0214_4cd8_87be_838ee702e3ca.slice/crio-c8f99bd9691a8a9a2b081087f18a52f0524339ee347479294470edd78dbb84bb WatchSource:0}: Error finding container c8f99bd9691a8a9a2b081087f18a52f0524339ee347479294470edd78dbb84bb: Status 404 returned error can't find the container with id c8f99bd9691a8a9a2b081087f18a52f0524339ee347479294470edd78dbb84bb Jul 08 04:18:10.942349 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.942327 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-flg5k"] Jul 08 04:18:10.945344 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:10.945315 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1affeec_0fb0_48c5_9c3c_3b3f701e12a2.slice/crio-1e772c20a63e9976e92e8d1a4f403e2dc414e338014946143596be3a413546f2 WatchSource:0}: Error finding container 1e772c20a63e9976e92e8d1a4f403e2dc414e338014946143596be3a413546f2: Status 404 returned error can't find the container with id 1e772c20a63e9976e92e8d1a4f403e2dc414e338014946143596be3a413546f2 Jul 08 04:18:10.978320 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.978248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:10.978320 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.978311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:10.978343 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978409 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978425 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978429 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978449 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978483 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.978466605 +0000 UTC m=+37.080305518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978488 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:10.978494 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978499 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.978491437 +0000 UTC m=+37.080330342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:10.978980 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:10.978544 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:11.978525368 +0000 UTC m=+37.080364279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:11.079158 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.079124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:11.079346 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.079261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:11.079346 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.079292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:11.079346 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079298 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:11.079503 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079359 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:12.079343768 +0000 UTC m=+37.181182673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:11.079503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.079386 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:11.079503 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079387 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:11.079503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.079464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:11.079503 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079485 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:12.07947129 +0000 UTC m=+37.181310203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:11.079780 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079527 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:11.079780 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079537 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:12.079520258 +0000 UTC m=+37.181359165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:11.079780 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079562 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:12.079552244 +0000 UTC m=+37.181391149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:11.079780 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079421 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:11.079780 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.079588 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:12.079582669 +0000 UTC m=+37.181421575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:11.439989 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.439956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:11.440391 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.440361 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:11.440723 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.440706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:11.443647 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.443622 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 04:18:11.443647 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.443644 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 04:18:11.443865 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.443621 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gp6ks\"" Jul 08 04:18:11.444071 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.444000 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbjls\"" Jul 08 04:18:11.645463 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.645423 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sdfj6" event={"ID":"7d1da944-0214-4cd8-87be-838ee702e3ca","Type":"ContainerStarted","Data":"c8f99bd9691a8a9a2b081087f18a52f0524339ee347479294470edd78dbb84bb"} Jul 08 04:18:11.648802 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.648705 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" event={"ID":"0997be05-260b-4d34-b609-24723de13d50","Type":"ContainerStarted","Data":"8cad3ce7dd06229c357198a5138300a561d36d8f9fd78f5147a7b2fdd883345a"} Jul 08 04:18:11.655013 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.654980 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" event={"ID":"4ecfabab-c5bf-425e-9d4e-a74550dd2754","Type":"ContainerStarted","Data":"fb6a467aaea69adb99c6893ddd414f271115bacecd10a71b85387ca19707c2c1"} Jul 08 04:18:11.659525 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.659496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" event={"ID":"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2","Type":"ContainerStarted","Data":"1e772c20a63e9976e92e8d1a4f403e2dc414e338014946143596be3a413546f2"} Jul 08 04:18:11.663406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.663373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" event={"ID":"10be44bf-2a06-440b-baf5-951568059b8c","Type":"ContainerStarted","Data":"ef104bd5d51222343a412ea7dbda4f2626fe5c44f08a80a633e43d311fff12fa"} Jul 08 04:18:11.670547 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.670509 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" event={"ID":"63781dc0-d73d-4209-9cd7-49951c8693eb","Type":"ContainerStarted","Data":"d0d5d68c877223fd7fb82466fef4d71da9057fadf8f46cf936e7d801ffe4a977"} Jul 08 04:18:11.673648 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.673619 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" event={"ID":"98ab92c6-780e-439c-b578-df03697fa3e1","Type":"ContainerStarted","Data":"6fe327048809b4ebd5c8b40fd1ef9499a19fe040cca2789a52fc095bf0cf2d5b"} Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.988925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.989006 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:11.989040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989274 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989343 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:13.989323791 +0000 UTC m=+39.091162698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989778 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989795 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989844 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:13.989828988 +0000 UTC m=+39.091667912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989908 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989917 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:11.989979 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:11.989946 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:13.989935753 +0000 UTC m=+39.091774674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:12.090813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:12.090862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:12.090975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:12.091005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:12.091067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091219 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:14.091200222 +0000 UTC m=+39.193039151 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091655 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091718 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:14.091700911 +0000 UTC m=+39.193539883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091798 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091831 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:14.091820428 +0000 UTC m=+39.193659334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091888 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091924 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:14.09191356 +0000 UTC m=+39.193752466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.091993 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:12.092082 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:12.092040 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:14.092028944 +0000 UTC m=+39.193867851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:13.687155 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:13.687106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" event={"ID":"0997be05-260b-4d34-b609-24723de13d50","Type":"ContainerStarted","Data":"d5d73fe63ed3f19c1b67aa3af98bf0c5e5ec0b4305257957946ec60ebe8e5c44"} Jul 08 04:18:13.705635 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:13.705577 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x88qs" podStartSLOduration=35.718332382 podStartE2EDuration="37.705557276s" totalImagesPullingTime="1.987224894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:13.704204376 +0000 UTC m=+38.806043305" watchObservedRunningTime="2026-07-08 04:18:13.705557276 +0000 UTC m=+38.807396206" Jul 08 04:18:13.813779 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:13.813717 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:13.904249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:13.904161 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:14.014333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.014263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:14.014333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.014320 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:14.014556 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014419 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:14.014556 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014435 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:14.014556 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014500 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.014483527 +0000 UTC m=+43.116322456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:14.014556 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014531 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:14.014857 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.014562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:14.014857 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014600 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.014581866 +0000 UTC m=+43.116420786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:14.014857 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014663 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:14.014857 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014675 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:14.014857 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.014725 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.014712704 +0000 UTC m=+43.116551610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:14.115236 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.115142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:14.115236 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.115188 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:14.115487 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115345 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:14.115487 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.115394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:14.115487 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115425 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.115404365 +0000 UTC m=+43.217243275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:14.115487 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.115469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:14.115487 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115484 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:14.115545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115559 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115612 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.115596512 +0000 UTC m=+43.217435418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115627 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115632 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.1156225 +0000 UTC m=+43.217461409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115665 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.115650954 +0000 UTC m=+43.217489865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:14.115745 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:14.115707 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:18.115692547 +0000 UTC m=+43.217531467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:18.053228 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.053186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.053246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.053268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053376 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053399 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053406 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053376 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053457 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053461 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.053443062 +0000 UTC m=+51.155281968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053483 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.05347253 +0000 UTC m=+51.155311439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:18.053699 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.053498 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.053489687 +0000 UTC m=+51.155328593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:18.154754 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.154713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:18.154963 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.154784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:18.154963 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.154882 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:18.154963 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.154900 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:18.154963 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.154952 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.154934656 +0000 UTC m=+51.256773564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:18.155148 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.154972 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.154962547 +0000 UTC m=+51.256801475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:18.155148 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.155076 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:18.155148 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.155109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:18.155289 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:18.155173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:18.155289 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.155203 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:18.155289 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.155256 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.155239866 +0000 UTC m=+51.257078780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:18.155289 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.155277 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:18.155289 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.155285 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.155271693 +0000 UTC m=+51.257110603 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:18.155471 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:18.155327 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:26.15531133 +0000 UTC m=+51.257150260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:19.416295 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:19.416152 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:19.475906 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:19.475866 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:20.709967 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.709371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" event={"ID":"4ecfabab-c5bf-425e-9d4e-a74550dd2754","Type":"ContainerStarted","Data":"129936a5711d7939ab80fa601f7fa75102e1a35f623e833063af99dbeb5f88ca"} Jul 08 04:18:20.713097 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.713047 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerStarted","Data":"3ae62915d3a6e0f87bda3d479c98b55dbf9a234dae72f16fbc7b7fe04d155760"} Jul 08 04:18:20.716484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.716316 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/0.log" Jul 08 04:18:20.716484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.716354 2582 generic.go:356] "Generic (PLEG): container finished" podID="d1affeec-0fb0-48c5-9c3c-3b3f701e12a2" containerID="c5177433ff82754f4a89ced9762989c767822749f0dd30eececbd9d082342606" exitCode=255 Jul 08 04:18:20.716484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.716429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" event={"ID":"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2","Type":"ContainerDied","Data":"c5177433ff82754f4a89ced9762989c767822749f0dd30eececbd9d082342606"} Jul 08 04:18:20.717190 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.716975 2582 scope.go:117] "RemoveContainer" containerID="c5177433ff82754f4a89ced9762989c767822749f0dd30eececbd9d082342606" Jul 08 04:18:20.721422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.720669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" event={"ID":"10be44bf-2a06-440b-baf5-951568059b8c","Type":"ContainerStarted","Data":"b441e8f8ae6a9c7fdcd4400da5629a2cb596d4ccf4dbd19c8726b82f1d230d15"} Jul 08 04:18:20.724772 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.724443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" event={"ID":"63781dc0-d73d-4209-9cd7-49951c8693eb","Type":"ContainerStarted","Data":"c9e4efec9d56093fdef50d8cf8624fb7e08dbc9e48238af2513f69fef87cc123"} Jul 08 04:18:20.729416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.729389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" event={"ID":"98ab92c6-780e-439c-b578-df03697fa3e1","Type":"ContainerStarted","Data":"ec6d0eba118b1658efdff4e12a9446629201f3d92808351a4c792a407be61edf"} Jul 08 04:18:20.744091 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.742496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sdfj6" event={"ID":"7d1da944-0214-4cd8-87be-838ee702e3ca","Type":"ContainerStarted","Data":"f3fa8d985b7412f69d75ec360952965d0b644196eff337d4493169a499e0a34c"} Jul 08 04:18:20.755847 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.750255 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" podStartSLOduration=35.358281138 podStartE2EDuration="44.750233957s" totalImagesPullingTime="9.391952819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:20.728733736 +0000 UTC m=+45.830572668" watchObservedRunningTime="2026-07-08 04:18:20.750233957 +0000 UTC m=+45.852072886" Jul 08 04:18:20.802985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.802242 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" podStartSLOduration=35.456709561 podStartE2EDuration="44.80221724s" totalImagesPullingTime="9.345507679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:20.801289874 +0000 UTC m=+45.903128804" watchObservedRunningTime="2026-07-08 04:18:20.80221724 +0000 UTC m=+45.904056169" Jul 08 04:18:20.843004 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.842537 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" podStartSLOduration=35.477615074 podStartE2EDuration="44.84251354s" totalImagesPullingTime="9.364898466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:20.822400761 +0000 UTC m=+45.924239690" watchObservedRunningTime="2026-07-08 04:18:20.84251354 +0000 UTC m=+45.944352471" Jul 08 04:18:20.843004 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.842863 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sdfj6" podStartSLOduration=1.514188845 podStartE2EDuration="10.842855358s" totalImagesPullingTime="9.328666513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:20.840722704 +0000 UTC m=+45.942561632" watchObservedRunningTime="2026-07-08 04:18:20.842855358 +0000 UTC m=+45.944694288" Jul 08 04:18:20.868004 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:20.866072 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l5bxr" podStartSLOduration=26.18017126 podStartE2EDuration="35.864824927s" totalImagesPullingTime="9.684653667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:20.864153859 +0000 UTC m=+45.965992788" watchObservedRunningTime="2026-07-08 04:18:20.864824927 +0000 UTC m=+45.966663855" Jul 08 04:18:21.535221 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.535185 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2l852"] Jul 08 04:18:21.560416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.560388 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2l852"] Jul 08 04:18:21.560563 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.560493 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.562981 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.562958 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 04:18:21.563230 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.563213 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6rbmt\"" Jul 08 04:18:21.563328 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.563308 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 04:18:21.564320 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.564298 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:21.597958 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.597928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.598115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.598008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a326a767-e4f2-4d44-b7a0-8ff632906bb3-crio-socket\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.598115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.598045 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a326a767-e4f2-4d44-b7a0-8ff632906bb3-data-volume\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.598115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.598087 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.598490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.598421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws4h2\" (UniqueName: \"kubernetes.io/projected/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-api-access-ws4h2\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.614596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.614565 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:21.699377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a326a767-e4f2-4d44-b7a0-8ff632906bb3-crio-socket\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699569 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a326a767-e4f2-4d44-b7a0-8ff632906bb3-data-volume\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699569 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699569 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws4h2\" (UniqueName: \"kubernetes.io/projected/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-api-access-ws4h2\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699722 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699631 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a326a767-e4f2-4d44-b7a0-8ff632906bb3-crio-socket\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699722 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:21.699677 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 04:18:21.699840 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:21.699746 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:22.199724885 +0000 UTC m=+47.301563796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : secret "insights-runtime-extractor-tls" not found Jul 08 04:18:21.699840 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699831 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a326a767-e4f2-4d44-b7a0-8ff632906bb3-data-volume\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699951 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.699836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.699951 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:21.699882 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 04:18:21.699951 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:21.699916 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:22.199905382 +0000 UTC m=+47.301744290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : configmap "kube-rbac-proxy" not found Jul 08 04:18:21.711567 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.711529 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws4h2\" (UniqueName: \"kubernetes.io/projected/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-api-access-ws4h2\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:21.749247 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.749205 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="3ae62915d3a6e0f87bda3d479c98b55dbf9a234dae72f16fbc7b7fe04d155760" exitCode=0 Jul 08 04:18:21.749413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.749288 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"3ae62915d3a6e0f87bda3d479c98b55dbf9a234dae72f16fbc7b7fe04d155760"} Jul 08 04:18:21.751060 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.751042 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:18:21.755007 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.754986 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/0.log" Jul 08 04:18:21.755136 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.755029 2582 generic.go:356] "Generic (PLEG): container finished" podID="d1affeec-0fb0-48c5-9c3c-3b3f701e12a2" containerID="35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0" exitCode=255 Jul 08 04:18:21.755198 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.755176 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" event={"ID":"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2","Type":"ContainerDied","Data":"35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0"} Jul 08 04:18:21.755255 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.755222 2582 scope.go:117] "RemoveContainer" containerID="c5177433ff82754f4a89ced9762989c767822749f0dd30eececbd9d082342606" Jul 08 04:18:21.772504 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:21.772484 2582 scope.go:117] "RemoveContainer" containerID="35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0" Jul 08 04:18:21.773072 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:21.773045 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-flg5k_openshift-console-operator(d1affeec-0fb0-48c5-9c3c-3b3f701e12a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" podUID="d1affeec-0fb0-48c5-9c3c-3b3f701e12a2" Jul 08 04:18:22.166952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.166871 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp"] Jul 08 04:18:22.170103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.170064 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:22.183360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.183331 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" Jul 08 04:18:22.184805 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.184776 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp"] Jul 08 04:18:22.186661 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.186639 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:22.186661 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.186652 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 04:18:22.186968 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.186953 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9qncm\"" Jul 08 04:18:22.206840 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.206817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:22.206952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.206881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdqjm\" (UniqueName: \"kubernetes.io/projected/a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a-kube-api-access-pdqjm\") pod \"migrator-d57d58cc9-28cmp\" (UID: \"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" Jul 08 04:18:22.206952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.206929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:22.207041 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:22.206969 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 04:18:22.207041 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:22.207017 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 04:18:22.207107 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:22.207020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:23.207006564 +0000 UTC m=+48.308845470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : secret "insights-runtime-extractor-tls" not found Jul 08 04:18:22.207107 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:22.207076 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:23.207063375 +0000 UTC m=+48.308902281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : configmap "kube-rbac-proxy" not found Jul 08 04:18:22.270311 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.270276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:22.307671 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.307631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdqjm\" (UniqueName: \"kubernetes.io/projected/a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a-kube-api-access-pdqjm\") pod \"migrator-d57d58cc9-28cmp\" (UID: \"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" Jul 08 04:18:22.318969 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.318928 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdqjm\" (UniqueName: \"kubernetes.io/projected/a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a-kube-api-access-pdqjm\") pod \"migrator-d57d58cc9-28cmp\" (UID: \"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" Jul 08 04:18:22.370647 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.370602 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:22.470884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.470775 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:22.494704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.494653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" Jul 08 04:18:22.608879 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.608674 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4m79_74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3/dns-node-resolver/0.log" Jul 08 04:18:22.657065 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.657035 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp"] Jul 08 04:18:22.661261 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:22.661231 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda094d7a2_ccf4_4e9b_b8c0_ddd766e59b9a.slice/crio-38b5ccd3be473b8377701d15dbca06bef94467a5e84b3d18bcaab9e96849d45e WatchSource:0}: Error finding container 38b5ccd3be473b8377701d15dbca06bef94467a5e84b3d18bcaab9e96849d45e: Status 404 returned error can't find the container with id 38b5ccd3be473b8377701d15dbca06bef94467a5e84b3d18bcaab9e96849d45e Jul 08 04:18:22.759749 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.759490 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" event={"ID":"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a","Type":"ContainerStarted","Data":"38b5ccd3be473b8377701d15dbca06bef94467a5e84b3d18bcaab9e96849d45e"} Jul 08 04:18:22.762725 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.762694 2582 generic.go:356] "Generic (PLEG): container finished" podID="db617134-01de-4ba0-97af-ffe6581dfc0e" containerID="abada924e021da7d0932ec7c39e0ed07ebabdf272666851142b5822ccb4d4707" exitCode=0 Jul 08 04:18:22.762860 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.762782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerDied","Data":"abada924e021da7d0932ec7c39e0ed07ebabdf272666851142b5822ccb4d4707"} Jul 08 04:18:22.764406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.764391 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:18:22.764691 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.764678 2582 scope.go:117] "RemoveContainer" containerID="35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0" Jul 08 04:18:22.764869 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:22.764851 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-flg5k_openshift-console-operator(d1affeec-0fb0-48c5-9c3c-3b3f701e12a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" podUID="d1affeec-0fb0-48c5-9c3c-3b3f701e12a2" Jul 08 04:18:22.914427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:22.914396 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:23.011846 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.011726 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:23.207686 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.207663 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msdnn_bacccd3a-40f9-4908-9eda-a4f4c930852c/node-ca/0.log" Jul 08 04:18:23.216087 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.216062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:23.216223 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.216186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:23.216223 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:23.216189 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 04:18:23.216291 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:23.216253 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:25.2162329 +0000 UTC m=+50.318071806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : configmap "kube-rbac-proxy" not found Jul 08 04:18:23.216330 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:23.216297 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 04:18:23.216362 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:23.216335 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:25.216324706 +0000 UTC m=+50.318163612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : secret "insights-runtime-extractor-tls" not found Jul 08 04:18:23.608562 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.608530 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sdfj6_7d1da944-0214-4cd8-87be-838ee702e3ca/serve-healthcheck-canary/0.log" Jul 08 04:18:23.773496 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.773461 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-676sm" event={"ID":"db617134-01de-4ba0-97af-ffe6581dfc0e","Type":"ContainerStarted","Data":"4f35f31135c2a0287d43d935362922c1a057fbeb36a88ba2f5157a86f0dbde02"} Jul 08 04:18:23.797548 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.797491 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-676sm" podStartSLOduration=16.882170881 podStartE2EDuration="48.797466819s" totalImagesPullingTime="31.915295938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:23.795060333 +0000 UTC m=+48.896899260" watchObservedRunningTime="2026-07-08 04:18:23.797466819 +0000 UTC m=+48.899305747" Jul 08 04:18:23.916449 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:23.916350 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.017119 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.017062 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.506512 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.506491 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xwwvx"] Jul 08 04:18:24.532806 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.532783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xwwvx"] Jul 08 04:18:24.532909 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.532863 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.535850 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.535825 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 04:18:24.535980 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.535866 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 04:18:24.536867 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.536848 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 04:18:24.537017 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.536940 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-w7gdc\"" Jul 08 04:18:24.537017 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.536974 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 04:18:24.619061 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.619022 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.629512 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.629488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m62cr\" (UniqueName: \"kubernetes.io/projected/ce5195a3-d763-46f6-9148-66dacaacdcfd-kube-api-access-m62cr\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.629597 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.629575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-key\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.629677 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.629662 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-cabundle\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.719196 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.719164 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.730357 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.730305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m62cr\" (UniqueName: \"kubernetes.io/projected/ce5195a3-d763-46f6-9148-66dacaacdcfd-kube-api-access-m62cr\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.730445 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.730366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-key\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.730581 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.730561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-cabundle\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.731207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.731188 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-cabundle\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.734059 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.734036 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ce5195a3-d763-46f6-9148-66dacaacdcfd-signing-key\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.738301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.738281 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m62cr\" (UniqueName: \"kubernetes.io/projected/ce5195a3-d763-46f6-9148-66dacaacdcfd-kube-api-access-m62cr\") pod \"service-ca-758f599c5-xwwvx\" (UID: \"ce5195a3-d763-46f6-9148-66dacaacdcfd\") " pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.779168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.779131 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" event={"ID":"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a","Type":"ContainerStarted","Data":"b894f83dc226aec833212f7a064ba05bd99eea9b5e10321fb266b4e7e4098821"} Jul 08 04:18:24.779536 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.779187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" event={"ID":"a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a","Type":"ContainerStarted","Data":"d74927c211b8db0409b8eded806b11e2d5f6f4d0d5d2350465d7102c2844d9ce"} Jul 08 04:18:24.793518 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.793463 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28cmp" podStartSLOduration=0.980179967 podStartE2EDuration="2.79344701s" totalImagesPullingTime="1.813267043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:24.793059083 +0000 UTC m=+49.894898011" watchObservedRunningTime="2026-07-08 04:18:24.79344701 +0000 UTC m=+49.895285937" Jul 08 04:18:24.843663 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.843629 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xwwvx" Jul 08 04:18:24.871254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.871220 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.929309 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.929277 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:24.986070 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:24.985994 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xwwvx"] Jul 08 04:18:24.989318 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:24.989288 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5195a3_d763_46f6_9148_66dacaacdcfd.slice/crio-8c59f42e1f6763a1e3b28df29e03804159788691529f2cd717aaf8dad28a1728 WatchSource:0}: Error finding container 8c59f42e1f6763a1e3b28df29e03804159788691529f2cd717aaf8dad28a1728: Status 404 returned error can't find the container with id 8c59f42e1f6763a1e3b28df29e03804159788691529f2cd717aaf8dad28a1728 Jul 08 04:18:25.235306 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.235273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:25.235479 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.235387 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:25.235479 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:25.235410 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 04:18:25.235559 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:25.235480 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:29.235459779 +0000 UTC m=+54.337298685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : configmap "kube-rbac-proxy" not found Jul 08 04:18:25.235559 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:25.235539 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 04:18:25.235641 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:25.235589 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:29.235573782 +0000 UTC m=+54.337412705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : secret "insights-runtime-extractor-tls" not found Jul 08 04:18:25.402549 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.402508 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:25.784135 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.784095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xwwvx" event={"ID":"ce5195a3-d763-46f6-9148-66dacaacdcfd","Type":"ContainerStarted","Data":"bf69ce5d3f4cca4a91c733aeaae25abf0e02240f4ceb50dc6dc99a1b13cd397b"} Jul 08 04:18:25.784603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.784157 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xwwvx" event={"ID":"ce5195a3-d763-46f6-9148-66dacaacdcfd","Type":"ContainerStarted","Data":"8c59f42e1f6763a1e3b28df29e03804159788691529f2cd717aaf8dad28a1728"} Jul 08 04:18:25.800092 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:25.800030 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xwwvx" podStartSLOduration=1.800016005 podStartE2EDuration="1.800016005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:25.799292852 +0000 UTC m=+50.901131779" watchObservedRunningTime="2026-07-08 04:18:25.800016005 +0000 UTC m=+50.901854931" Jul 08 04:18:26.144831 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.144718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:26.144831 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.144810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:26.144831 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.144831 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144892 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144912 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d7b8c546-xx4lq: secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144945 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144958 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls podName:ec857cc2-e1ed-4045-be8e-93a05523ab8e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.144944749 +0000 UTC m=+67.246783659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls") pod "image-registry-76d7b8c546-xx4lq" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e") : secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144962 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbfd85b89-cjqcv: secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.144971 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.145009 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls podName:de847b82-b567-41b9-9f9e-92ee2815ca85 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.144993646 +0000 UTC m=+67.246832555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls") pod "image-registry-7cbfd85b89-cjqcv" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85") : secret "image-registry-tls" not found Jul 08 04:18:26.145104 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.145023 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls podName:ca7f6469-7920-4c58-92be-eb8dc3791202 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.145016347 +0000 UTC m=+67.246855252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52c44" (UID: "ca7f6469-7920-4c58-92be-eb8dc3791202") : secret "cluster-monitoring-operator-tls" not found Jul 08 04:18:26.249458 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.249414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:26.249671 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.249490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:26.249729 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.249588 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 04:18:26.250436 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250413 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 04:18:26.250520 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.249679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:26.250520 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250420 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert podName:f798bae3-92ec-4263-addb-28e02abf35f4 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.250398844 +0000 UTC m=+67.352237765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-skrm2" (UID: "f798bae3-92ec-4263-addb-28e02abf35f4") : secret "networking-console-plugin-cert" not found Jul 08 04:18:26.250520 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250487 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.250474184 +0000 UTC m=+67.352313090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : configmap references non-existent config key: service-ca.crt Jul 08 04:18:26.250520 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250509 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs podName:e0fd4931-ebcc-4948-b5fd-748844c3c538 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.250498611 +0000 UTC m=+67.352337523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs") pod "router-default-bdfc5556f-2c48c" (UID: "e0fd4931-ebcc-4948-b5fd-748844c3c538") : secret "router-metrics-certs-default" not found Jul 08 04:18:26.250687 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.250577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:26.250687 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.250607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:26.250687 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250650 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 04:18:26.250853 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250700 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls podName:51e092a6-2a2d-4df1-8be9-81078a8081c1 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.250687133 +0000 UTC m=+67.352526043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-89vwl" (UID: "51e092a6-2a2d-4df1-8be9-81078a8081c1") : secret "samples-operator-tls" not found Jul 08 04:18:26.250853 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250790 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 04:18:26.250853 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:26.250842 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls podName:0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e nodeName:}" failed. No retries permitted until 2026-07-08 04:18:42.250827731 +0000 UTC m=+67.352666650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls") pod "dns-default-7xcmg" (UID: "0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e") : secret "dns-default-metrics-tls" not found Jul 08 04:18:26.483567 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.483437 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:26.585312 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.585275 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:26.775934 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.775900 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:26.875985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.875941 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:26.974861 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:26.974816 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:27.428038 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:27.427998 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:28.002217 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:28.002182 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:28.287053 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:28.286953 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:28.413233 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:28.413188 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:28.792046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:28.792010 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:29.178235 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:29.178152 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:29.281388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:29.281354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:29.281540 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:29.281483 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 04:18:29.281594 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:29.281482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:29.281594 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:29.281545 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:37.281528624 +0000 UTC m=+62.383367537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : configmap "kube-rbac-proxy" not found Jul 08 04:18:29.281698 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:29.281559 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 04:18:29.281698 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:29.281652 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls podName:a326a767-e4f2-4d44-b7a0-8ff632906bb3 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:37.281635433 +0000 UTC m=+62.383474340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l852" (UID: "a326a767-e4f2-4d44-b7a0-8ff632906bb3") : secret "insights-runtime-extractor-tls" not found Jul 08 04:18:29.579617 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:29.579581 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:29.784595 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:29.784560 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:30.174617 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.174576 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:30.223944 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.223894 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:30.525238 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.525190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:30.625872 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.625832 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:30.687999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.687958 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:30.688143 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.688034 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:30.688398 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.688386 2582 scope.go:117] "RemoveContainer" containerID="35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0" Jul 08 04:18:30.688556 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:30.688537 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-flg5k_openshift-console-operator(d1affeec-0fb0-48c5-9c3c-3b3f701e12a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" podUID="d1affeec-0fb0-48c5-9c3c-3b3f701e12a2" Jul 08 04:18:30.725683 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:30.725651 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:31.177977 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:31.177942 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:31.278143 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:31.278111 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:31.579144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:31.579088 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:31.678315 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:31.678274 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:31.981025 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:31.980922 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:32.383363 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:32.383329 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:32.579200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:32.579161 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:32.700265 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:32.700175 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:33.176065 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:33.176030 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:33.775863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:33.775828 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:33.989438 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:33.989384 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:34.379909 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:34.379868 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:34.581146 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:34.581114 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:34.806954 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:34.806916 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:35.178235 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:35.178133 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:35.573137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:35.573099 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:35.979585 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:35.979479 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:36.181715 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:36.181681 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:36.578164 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:36.578132 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:36.640140 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:36.640109 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-skwn9" Jul 08 04:18:36.984123 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:36.984028 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:37.177813 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.177778 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:37.362340 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.362299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:37.362528 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.362426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:37.362870 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.362854 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a326a767-e4f2-4d44-b7a0-8ff632906bb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:37.364681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.364661 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a326a767-e4f2-4d44-b7a0-8ff632906bb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l852\" (UID: \"a326a767-e4f2-4d44-b7a0-8ff632906bb3\") " pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:37.473477 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.473448 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6rbmt\"" Jul 08 04:18:37.481328 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.481306 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2l852" Jul 08 04:18:37.604075 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.604055 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2l852"] Jul 08 04:18:37.606603 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:37.606569 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda326a767_e4f2_4d44_b7a0_8ff632906bb3.slice/crio-22b978a1422b80b028cabbc129802635ff27702a8ecccb3e34e71f55e14d5ce1 WatchSource:0}: Error finding container 22b978a1422b80b028cabbc129802635ff27702a8ecccb3e34e71f55e14d5ce1: Status 404 returned error can't find the container with id 22b978a1422b80b028cabbc129802635ff27702a8ecccb3e34e71f55e14d5ce1 Jul 08 04:18:37.824985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:37.824950 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l852" event={"ID":"a326a767-e4f2-4d44-b7a0-8ff632906bb3","Type":"ContainerStarted","Data":"22b978a1422b80b028cabbc129802635ff27702a8ecccb3e34e71f55e14d5ce1"} Jul 08 04:18:39.084928 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:39.084886 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:39.141168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:39.141136 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:40.189351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.189304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:40.189882 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.189450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:40.192165 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.192135 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 04:18:40.192165 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.192135 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 04:18:40.202677 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.202638 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5906284-7602-4e4c-b6ad-e316355875b4-metrics-certs\") pod \"network-metrics-daemon-dzbws\" (UID: \"b5906284-7602-4e4c-b6ad-e316355875b4\") " pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:40.202977 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.202954 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75400726-ae72-406b-b112-64e25c26f5d7-original-pull-secret\") pod \"global-pull-secret-syncer-n7lnz\" (UID: \"75400726-ae72-406b-b112-64e25c26f5d7\") " pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:40.267001 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.266967 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gp6ks\"" Jul 08 04:18:40.275099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.275065 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzbws" Jul 08 04:18:40.309587 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.309554 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lnz" Jul 08 04:18:40.391925 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.391889 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:40.394518 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.394489 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrds5\" (UniqueName: \"kubernetes.io/projected/7117c335-b1a4-4d8a-b896-51919001f08c-kube-api-access-lrds5\") pod \"network-check-target-vjp7n\" (UID: \"7117c335-b1a4-4d8a-b896-51919001f08c\") " pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:40.558793 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.558539 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbjls\"" Jul 08 04:18:40.566567 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:40.566533 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:42.210410 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.210181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:42.210410 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.210257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:42.210410 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.210296 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:42.213987 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.213637 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca7f6469-7920-4c58-92be-eb8dc3791202-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52c44\" (UID: \"ca7f6469-7920-4c58-92be-eb8dc3791202\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:42.213987 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.213642 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"image-registry-7cbfd85b89-cjqcv\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:42.215231 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.215178 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"image-registry-76d7b8c546-xx4lq\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:42.264265 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.264238 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjp7n"] Jul 08 04:18:42.267590 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.267553 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7117c335_b1a4_4d8a_b896_51919001f08c.slice/crio-cae2048399eea892fd3ddae453b70599371a7f7b3624f08c971c94c64d4b621d WatchSource:0}: Error finding container cae2048399eea892fd3ddae453b70599371a7f7b3624f08c971c94c64d4b621d: Status 404 returned error can't find the container with id cae2048399eea892fd3ddae453b70599371a7f7b3624f08c971c94c64d4b621d Jul 08 04:18:42.312236 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.312203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:42.312394 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.312248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:42.312394 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.312302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:42.312394 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.312368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:42.312584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.312558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:42.313161 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.313130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0fd4931-ebcc-4948-b5fd-748844c3c538-service-ca-bundle\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:42.314700 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.314658 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f798bae3-92ec-4263-addb-28e02abf35f4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-skrm2\" (UID: \"f798bae3-92ec-4263-addb-28e02abf35f4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:42.314981 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.314962 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e-metrics-tls\") pod \"dns-default-7xcmg\" (UID: \"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e\") " pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:42.315069 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.314986 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51e092a6-2a2d-4df1-8be9-81078a8081c1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-89vwl\" (UID: \"51e092a6-2a2d-4df1-8be9-81078a8081c1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:42.315069 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.314985 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0fd4931-ebcc-4948-b5fd-748844c3c538-metrics-certs\") pod \"router-default-bdfc5556f-2c48c\" (UID: \"e0fd4931-ebcc-4948-b5fd-748844c3c538\") " pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:42.396734 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.396648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p4jt\"" Jul 08 04:18:42.404631 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.404608 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:42.431804 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.431778 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:42.450695 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.450666 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jlbqq\"" Jul 08 04:18:42.459184 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.459158 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" Jul 08 04:18:42.466717 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.466648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q8xbn\"" Jul 08 04:18:42.472124 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.472086 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d5gcx\"" Jul 08 04:18:42.475099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.475069 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" Jul 08 04:18:42.480026 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.480000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:42.491193 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.490862 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dzbws"] Jul 08 04:18:42.491193 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.491123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8jn4l\"" Jul 08 04:18:42.494459 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.494220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" Jul 08 04:18:42.496137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.494783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7lnz"] Jul 08 04:18:42.496137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.495728 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xh8sv\"" Jul 08 04:18:42.501915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.501877 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:42.564122 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.561729 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde847b82_b567_41b9_9f9e_92ee2815ca85.slice/crio-c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934 WatchSource:0}: Error finding container c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934: Status 404 returned error can't find the container with id c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934 Jul 08 04:18:42.564122 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.564019 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:18:42.662480 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.661429 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:18:42.700636 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.700600 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44"] Jul 08 04:18:42.703816 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.703597 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7f6469_7920_4c58_92be_eb8dc3791202.slice/crio-2a3bf02f5dc1194455a9f68198e1306aa04c2ba0ab0ebeb2f12f8fd11a47088e WatchSource:0}: Error finding container 2a3bf02f5dc1194455a9f68198e1306aa04c2ba0ab0ebeb2f12f8fd11a47088e: Status 404 returned error can't find the container with id 2a3bf02f5dc1194455a9f68198e1306aa04c2ba0ab0ebeb2f12f8fd11a47088e Jul 08 04:18:42.727068 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.726210 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl"] Jul 08 04:18:42.748455 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.747143 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-skrm2"] Jul 08 04:18:42.753227 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.752999 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf798bae3_92ec_4263_addb_28e02abf35f4.slice/crio-4fa6babe21fb078a662015f8c5eb87cf397788b0b96d99c0610e41cf8496dee8 WatchSource:0}: Error finding container 4fa6babe21fb078a662015f8c5eb87cf397788b0b96d99c0610e41cf8496dee8: Status 404 returned error can't find the container with id 4fa6babe21fb078a662015f8c5eb87cf397788b0b96d99c0610e41cf8496dee8 Jul 08 04:18:42.772403 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.772375 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xcmg"] Jul 08 04:18:42.775607 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.775577 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c981a2f_745f_4eb5_bac7_22ecbbfa8c6e.slice/crio-dde5f3ddcb5f69a63eedc9b5f575e062a285f688415b3e9c6d008f0a0ad29a4c WatchSource:0}: Error finding container dde5f3ddcb5f69a63eedc9b5f575e062a285f688415b3e9c6d008f0a0ad29a4c: Status 404 returned error can't find the container with id dde5f3ddcb5f69a63eedc9b5f575e062a285f688415b3e9c6d008f0a0ad29a4c Jul 08 04:18:42.846081 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.846042 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xcmg" event={"ID":"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e","Type":"ContainerStarted","Data":"dde5f3ddcb5f69a63eedc9b5f575e062a285f688415b3e9c6d008f0a0ad29a4c"} Jul 08 04:18:42.847359 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.847326 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" event={"ID":"f798bae3-92ec-4263-addb-28e02abf35f4","Type":"ContainerStarted","Data":"4fa6babe21fb078a662015f8c5eb87cf397788b0b96d99c0610e41cf8496dee8"} Jul 08 04:18:42.849106 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.849076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" event={"ID":"ec857cc2-e1ed-4045-be8e-93a05523ab8e","Type":"ContainerStarted","Data":"9a542bc5c2097441710ce179e868888ede286b3da71a13f36444c8be8cb7e00b"} Jul 08 04:18:42.849341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.849127 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" event={"ID":"ec857cc2-e1ed-4045-be8e-93a05523ab8e","Type":"ContainerStarted","Data":"4be81ae5118fdf8f820ba76c35973fcc9eb0252da2292d206a55161fc0b22f74"} Jul 08 04:18:42.849341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.849156 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:18:42.851047 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.850736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7lnz" event={"ID":"75400726-ae72-406b-b112-64e25c26f5d7","Type":"ContainerStarted","Data":"2c39758ba8831072bf66b1a4766559734d5ab0fff55bd8a243e0da04e0a07803"} Jul 08 04:18:42.852263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.852153 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" event={"ID":"51e092a6-2a2d-4df1-8be9-81078a8081c1","Type":"ContainerStarted","Data":"ecbb8daf9e8285d2ee75753c73cc22b228b8dc876160b0b3b8da3185e64e25b1"} Jul 08 04:18:42.853462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.853437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzbws" event={"ID":"b5906284-7602-4e4c-b6ad-e316355875b4","Type":"ContainerStarted","Data":"74734bec96854497873b9112114e0a36be402c45a5d8c90c1e974cbad48cf1a8"} Jul 08 04:18:42.855270 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.855246 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l852" event={"ID":"a326a767-e4f2-4d44-b7a0-8ff632906bb3","Type":"ContainerStarted","Data":"0955e7746a41f331cb6fb4141d9367a67f8634835fc3b093f23b02b3e95b1dc7"} Jul 08 04:18:42.856818 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.856791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" event={"ID":"ca7f6469-7920-4c58-92be-eb8dc3791202","Type":"ContainerStarted","Data":"2a3bf02f5dc1194455a9f68198e1306aa04c2ba0ab0ebeb2f12f8fd11a47088e"} Jul 08 04:18:42.858493 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.858472 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" event={"ID":"de847b82-b567-41b9-9f9e-92ee2815ca85","Type":"ContainerStarted","Data":"33b20395acea1b0c9b42768b3aad35806a4f67ed32cc966ced8f64bb12603730"} Jul 08 04:18:42.858577 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.858513 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" event={"ID":"de847b82-b567-41b9-9f9e-92ee2815ca85","Type":"ContainerStarted","Data":"c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934"} Jul 08 04:18:42.858639 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.858619 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:18:42.866713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.861274 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjp7n" event={"ID":"7117c335-b1a4-4d8a-b896-51919001f08c","Type":"ContainerStarted","Data":"858c9b1b9c339dcdaf43165d96bbdbcc53d48e7b2d528837239366279bb9990b"} Jul 08 04:18:42.866713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.861345 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjp7n" event={"ID":"7117c335-b1a4-4d8a-b896-51919001f08c","Type":"ContainerStarted","Data":"cae2048399eea892fd3ddae453b70599371a7f7b3624f08c971c94c64d4b621d"} Jul 08 04:18:42.866713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.861985 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:18:42.869297 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.868744 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" podStartSLOduration=66.868726266 podStartE2EDuration="1m6.868726266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:42.868481129 +0000 UTC m=+67.970320082" watchObservedRunningTime="2026-07-08 04:18:42.868726266 +0000 UTC m=+67.970565197" Jul 08 04:18:42.887343 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.887276 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" podStartSLOduration=67.887254451 podStartE2EDuration="1m7.887254451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:42.885465969 +0000 UTC m=+67.987304920" watchObservedRunningTime="2026-07-08 04:18:42.887254451 +0000 UTC m=+67.989093379" Jul 08 04:18:42.901386 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.901321 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vjp7n" podStartSLOduration=67.90129846 podStartE2EDuration="1m7.90129846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:42.900320117 +0000 UTC m=+68.002159063" watchObservedRunningTime="2026-07-08 04:18:42.90129846 +0000 UTC m=+68.003137388" Jul 08 04:18:42.976788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:42.976747 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bdfc5556f-2c48c"] Jul 08 04:18:42.979555 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:42.979529 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0fd4931_ebcc_4948_b5fd_748844c3c538.slice/crio-13858cb22773a571eadbfe46abebfac1eb29166f1766c176374f9e30d87d321a WatchSource:0}: Error finding container 13858cb22773a571eadbfe46abebfac1eb29166f1766c176374f9e30d87d321a: Status 404 returned error can't find the container with id 13858cb22773a571eadbfe46abebfac1eb29166f1766c176374f9e30d87d321a Jul 08 04:18:43.132229 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.131662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5"] Jul 08 04:18:43.138272 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.138245 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb"] Jul 08 04:18:43.138435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.138407 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.141088 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141062 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 04:18:43.141201 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141132 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 04:18:43.141291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141276 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w65sd\"" Jul 08 04:18:43.141372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141301 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 04:18:43.141372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141326 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 04:18:43.141619 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141596 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg"] Jul 08 04:18:43.141717 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.141654 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.144023 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.144005 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 04:18:43.145901 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.145877 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.146526 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.146504 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:18:43.147925 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.147906 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5"] Jul 08 04:18:43.149168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.148719 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 04:18:43.149168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.149010 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 04:18:43.149168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.149061 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 04:18:43.149527 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.149511 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 04:18:43.150148 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.149965 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb"] Jul 08 04:18:43.165921 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.165891 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg"] Jul 08 04:18:43.224558 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.224480 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n"] Jul 08 04:18:43.227225 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.227193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7thtw\" (UniqueName: \"kubernetes.io/projected/c5af89fe-015c-4a95-8b1a-7560c390b7ed-kube-api-access-7thtw\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.227424 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.227266 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5af89fe-015c-4a95-8b1a-7560c390b7ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.229085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.229062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.241405 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.241374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n"] Jul 08 04:18:43.249880 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.249833 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:43.328530 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.328530 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328529 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-registry-certificates\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ccca29-78e6-4d63-b2b6-0f3642507a08-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328623 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5af89fe-015c-4a95-8b1a-7560c390b7ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328663 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6m5d\" (UniqueName: \"kubernetes.io/projected/f8ccca29-78e6-4d63-b2b6-0f3642507a08-kube-api-access-z6m5d\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-klusterlet-config\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328747 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-image-registry-private-configuration\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328800 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.328824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-installation-pull-secrets\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328851 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-trusted-ca\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-tmp\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328916 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-registry-tls\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328940 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-bound-sa-token\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.328972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh229\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-kube-api-access-gh229\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.329006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c2qv\" (UniqueName: \"kubernetes.io/projected/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-kube-api-access-4c2qv\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.329047 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.329080 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7thtw\" (UniqueName: \"kubernetes.io/projected/c5af89fe-015c-4a95-8b1a-7560c390b7ed-kube-api-access-7thtw\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.329291 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.329107 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74bde0a9-b245-412b-ad44-50d81afb20df-ca-trust-extracted\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.332630 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.332600 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5af89fe-015c-4a95-8b1a-7560c390b7ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.342364 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.342334 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7thtw\" (UniqueName: \"kubernetes.io/projected/c5af89fe-015c-4a95-8b1a-7560c390b7ed-kube-api-access-7thtw\") pod \"managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5\" (UID: \"c5af89fe-015c-4a95-8b1a-7560c390b7ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.352113 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.352056 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430566 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-image-registry-private-configuration\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-installation-pull-secrets\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430673 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-trusted-ca\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-tmp\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-registry-tls\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.431191 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.430748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-bound-sa-token\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.432205 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432047 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-tmp\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh229\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-kube-api-access-gh229\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c2qv\" (UniqueName: \"kubernetes.io/projected/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-kube-api-access-4c2qv\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74bde0a9-b245-412b-ad44-50d81afb20df-ca-trust-extracted\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-registry-certificates\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ccca29-78e6-4d63-b2b6-0f3642507a08-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432648 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6m5d\" (UniqueName: \"kubernetes.io/projected/f8ccca29-78e6-4d63-b2b6-0f3642507a08-kube-api-access-z6m5d\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.433388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.432680 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-klusterlet-config\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.434359 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.434006 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-trusted-ca\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.434453 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.434440 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74bde0a9-b245-412b-ad44-50d81afb20df-registry-certificates\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.434629 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.434607 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74bde0a9-b245-412b-ad44-50d81afb20df-ca-trust-extracted\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.451629 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.451026 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh229\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-kube-api-access-gh229\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.451780 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.451589 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.454484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.454412 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8ccca29-78e6-4d63-b2b6-0f3642507a08-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.455561 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.455515 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.455794 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.455772 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-installation-pull-secrets\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.456111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.456089 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74bde0a9-b245-412b-ad44-50d81afb20df-image-registry-private-configuration\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.456269 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.456101 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c2qv\" (UniqueName: \"kubernetes.io/projected/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-kube-api-access-4c2qv\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.457530 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.457484 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.457530 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.457508 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6m5d\" (UniqueName: \"kubernetes.io/projected/f8ccca29-78e6-4d63-b2b6-0f3642507a08-kube-api-access-z6m5d\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.458526 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.458501 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f8ccca29-78e6-4d63-b2b6-0f3642507a08-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c75b5f46d-ppskg\" (UID: \"f8ccca29-78e6-4d63-b2b6-0f3642507a08\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.458823 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.458805 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-registry-tls\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.459276 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.459254 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de-klusterlet-config\") pod \"klusterlet-addon-workmgr-69cd45c9df-hfcsb\" (UID: \"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.460399 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.460276 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bde0a9-b245-412b-ad44-50d81afb20df-bound-sa-token\") pod \"image-registry-7b8f8b8cb4-9dg6n\" (UID: \"74bde0a9-b245-412b-ad44-50d81afb20df\") " pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.469514 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.469400 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" Jul 08 04:18:43.479784 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.479719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:43.488070 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.487853 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" Jul 08 04:18:43.549239 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.548807 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:43.698840 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.698771 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5"] Jul 08 04:18:43.701332 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:43.701292 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5af89fe_015c_4a95_8b1a_7560c390b7ed.slice/crio-3631affdd29533da7edea1209b5ca5b2578c966e331ecaad2e41709476479da9 WatchSource:0}: Error finding container 3631affdd29533da7edea1209b5ca5b2578c966e331ecaad2e41709476479da9: Status 404 returned error can't find the container with id 3631affdd29533da7edea1209b5ca5b2578c966e331ecaad2e41709476479da9 Jul 08 04:18:43.734463 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.734017 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb"] Jul 08 04:18:43.766509 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.766431 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg"] Jul 08 04:18:43.771644 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:43.771612 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8ccca29_78e6_4d63_b2b6_0f3642507a08.slice/crio-fc41d7e8eec2f68741fe5c679f9746af3e47bcbdebb724cfa376d8f92f935380 WatchSource:0}: Error finding container fc41d7e8eec2f68741fe5c679f9746af3e47bcbdebb724cfa376d8f92f935380: Status 404 returned error can't find the container with id fc41d7e8eec2f68741fe5c679f9746af3e47bcbdebb724cfa376d8f92f935380 Jul 08 04:18:43.802118 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.802013 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n"] Jul 08 04:18:43.890128 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.890057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" event={"ID":"f8ccca29-78e6-4d63-b2b6-0f3642507a08","Type":"ContainerStarted","Data":"fc41d7e8eec2f68741fe5c679f9746af3e47bcbdebb724cfa376d8f92f935380"} Jul 08 04:18:43.896915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.896874 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bdfc5556f-2c48c" event={"ID":"e0fd4931-ebcc-4948-b5fd-748844c3c538","Type":"ContainerStarted","Data":"cbd658810bb9436e6e1fa58a20b9844be7a9fed459fe678a68979b019ecd5d92"} Jul 08 04:18:43.897088 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.896980 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bdfc5556f-2c48c" event={"ID":"e0fd4931-ebcc-4948-b5fd-748844c3c538","Type":"ContainerStarted","Data":"13858cb22773a571eadbfe46abebfac1eb29166f1766c176374f9e30d87d321a"} Jul 08 04:18:43.904499 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.904459 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l852" event={"ID":"a326a767-e4f2-4d44-b7a0-8ff632906bb3","Type":"ContainerStarted","Data":"4238bcc304c6b128a7ff53e0b9ae89ad340d3989ad78b1352a20105e9fba8593"} Jul 08 04:18:43.910197 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.910143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" event={"ID":"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de","Type":"ContainerStarted","Data":"710a9196c8234e9bf346e93020315e1ab93677455919f63ae13324f2d0f3d8f1"} Jul 08 04:18:43.913809 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:43.913293 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" event={"ID":"c5af89fe-015c-4a95-8b1a-7560c390b7ed","Type":"ContainerStarted","Data":"3631affdd29533da7edea1209b5ca5b2578c966e331ecaad2e41709476479da9"} Jul 08 04:18:44.337661 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:44.337610 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bde0a9_b245_412b_ad44_50d81afb20df.slice/crio-77830550ba8d162fc92e9c6e062360419dd4dbd9f7bf29f0c1f466d54a4cc989 WatchSource:0}: Error finding container 77830550ba8d162fc92e9c6e062360419dd4dbd9f7bf29f0c1f466d54a4cc989: Status 404 returned error can't find the container with id 77830550ba8d162fc92e9c6e062360419dd4dbd9f7bf29f0c1f466d54a4cc989 Jul 08 04:18:44.440773 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.440726 2582 scope.go:117] "RemoveContainer" containerID="35e8b20310542d1007ebfeaeeb9d5c8da952f53e638e87b67ee5333f8374efd0" Jul 08 04:18:44.480932 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.480899 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:44.484122 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.484097 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:44.504333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.502927 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-bdfc5556f-2c48c" podStartSLOduration=68.502908948 podStartE2EDuration="1m8.502908948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:43.919659763 +0000 UTC m=+69.021498707" watchObservedRunningTime="2026-07-08 04:18:44.502908948 +0000 UTC m=+69.604747877" Jul 08 04:18:44.919735 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.919700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" event={"ID":"74bde0a9-b245-412b-ad44-50d81afb20df","Type":"ContainerStarted","Data":"77830550ba8d162fc92e9c6e062360419dd4dbd9f7bf29f0c1f466d54a4cc989"} Jul 08 04:18:44.920398 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.920374 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:44.922002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:44.921975 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-bdfc5556f-2c48c" Jul 08 04:18:47.876416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:47.876308 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:47.962800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:47.962743 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:48.042013 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:48.041963 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:48.102044 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:48.102007 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:48.177349 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:48.177275 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:48.251140 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:48.251105 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:48.351636 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:48.351590 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.053647 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.053609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.110240 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.110204 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.474634 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.474597 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.561903 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.561871 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.630362 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.630323 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:49.731992 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:49.731889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:50.252549 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:50.252512 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:50.353236 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:50.353192 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:50.652521 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:50.652421 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:50.706811 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:50.706777 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:55.975928 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.975858 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:18:55.976419 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.975965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" event={"ID":"d1affeec-0fb0-48c5-9c3c-3b3f701e12a2","Type":"ContainerStarted","Data":"7a1e90863da3ddd3cee7545b20aae7f3df150e991f86c60c1d7de80966d05ce4"} Jul 08 04:18:55.976653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.976572 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:55.978152 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.978102 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xcmg" event={"ID":"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e","Type":"ContainerStarted","Data":"a29ec754f10fc364d2dbc059ad07cbd9e7ab8fe72b45a15336a57d6f29d9fcfb"} Jul 08 04:18:55.980615 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.980561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" event={"ID":"f798bae3-92ec-4263-addb-28e02abf35f4","Type":"ContainerStarted","Data":"17b4ab144f9017b1833c287bd62335ac6a937dd54e0af832b3fb547d7ec5664a"} Jul 08 04:18:55.982335 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.982304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzbws" event={"ID":"b5906284-7602-4e4c-b6ad-e316355875b4","Type":"ContainerStarted","Data":"b4a77711dfc3fd89ba484b27e8cebe6e80272a3d2143d9f516dcda34a987bc38"} Jul 08 04:18:55.985043 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.985021 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l852" event={"ID":"a326a767-e4f2-4d44-b7a0-8ff632906bb3","Type":"ContainerStarted","Data":"42b8ecaf0103da6e8505da882166819418824d7339c79d8f46642d7c0dee6ac4"} Jul 08 04:18:55.987082 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.987061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" event={"ID":"74bde0a9-b245-412b-ad44-50d81afb20df","Type":"ContainerStarted","Data":"1c22800ae4e780c94ffc741a5b1bf8e778c9c5c421fffe4d07b7dd4d6c3f4b8f"} Jul 08 04:18:55.987224 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.987212 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:18:55.995959 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:55.995905 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" podStartSLOduration=70.737938482 podStartE2EDuration="1m19.995887758s" totalImagesPullingTime="9.257949276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:55.994380266 +0000 UTC m=+81.096219196" watchObservedRunningTime="2026-07-08 04:18:55.995887758 +0000 UTC m=+81.097726687" Jul 08 04:18:56.009277 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.009224 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-skrm2" podStartSLOduration=55.283697233 podStartE2EDuration="1m6.009204256s" totalImagesPullingTime="10.725507023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:56.008620566 +0000 UTC m=+81.110459498" watchObservedRunningTime="2026-07-08 04:18:56.009204256 +0000 UTC m=+81.111043162" Jul 08 04:18:56.027028 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.026956 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2l852" podStartSLOduration=17.063102096 podStartE2EDuration="35.026934316s" totalImagesPullingTime="17.96383222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:56.026415421 +0000 UTC m=+81.128254348" watchObservedRunningTime="2026-07-08 04:18:56.026934316 +0000 UTC m=+81.128773243" Jul 08 04:18:56.047490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.047350 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" podStartSLOduration=13.047328775 podStartE2EDuration="13.047328775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:56.046824698 +0000 UTC m=+81.148663650" watchObservedRunningTime="2026-07-08 04:18:56.047328775 +0000 UTC m=+81.149167704" Jul 08 04:18:56.140392 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.140274 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.244275 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.241215 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.478388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.478349 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.579067 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.579035 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.648948 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.648910 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.649128 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.649010 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2"] Jul 08 04:18:56.667707 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.667684 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2"] Jul 08 04:18:56.667876 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.667805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:56.670646 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.670409 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 04:18:56.670646 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.670424 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8rl7l\"" Jul 08 04:18:56.755258 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.755172 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-flg5k" Jul 08 04:18:56.773944 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.773869 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bff7085f-f8fa-4b10-9dc4-1b432d93cc2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jtvv2\" (UID: \"bff7085f-f8fa-4b10-9dc4-1b432d93cc2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:56.808330 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.808295 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.875678 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.875639 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bff7085f-f8fa-4b10-9dc4-1b432d93cc2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jtvv2\" (UID: \"bff7085f-f8fa-4b10-9dc4-1b432d93cc2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:56.878958 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.878924 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bff7085f-f8fa-4b10-9dc4-1b432d93cc2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jtvv2\" (UID: \"bff7085f-f8fa-4b10-9dc4-1b432d93cc2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:56.927621 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.927587 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-m8q57"] Jul 08 04:18:56.932391 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.932366 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:18:56.935155 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.935130 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 04:18:56.935155 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.935142 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 04:18:56.935680 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.935663 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7bz8f\"" Jul 08 04:18:56.940680 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.940389 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-m8q57"] Jul 08 04:18:56.951563 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.951528 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:56.976560 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.976519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26z57\" (UniqueName: \"kubernetes.io/projected/08bcdf13-ab24-43df-93cb-624f8586e50e-kube-api-access-26z57\") pod \"downloads-685bfb5f64-m8q57\" (UID: \"08bcdf13-ab24-43df-93cb-624f8586e50e\") " pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:18:56.978413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.978391 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:56.993202 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.993171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7lnz" event={"ID":"75400726-ae72-406b-b112-64e25c26f5d7","Type":"ContainerStarted","Data":"8be1bbc32d8a3973d6fe9405b510a4f2a317b788c76da3ce1ceb158cd824ae86"} Jul 08 04:18:56.995357 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.995330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" event={"ID":"f8ccca29-78e6-4d63-b2b6-0f3642507a08","Type":"ContainerStarted","Data":"c606706112b1e63941c5a937d8b842cabed8e8b12600034d2c4f087e6f164a70"} Jul 08 04:18:56.995513 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.995370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" event={"ID":"f8ccca29-78e6-4d63-b2b6-0f3642507a08","Type":"ContainerStarted","Data":"5599803440dfbd3858a98dee9c8b386ec4b695a4bd16d10959dd71fcb59089ec"} Jul 08 04:18:56.995513 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.995387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" event={"ID":"f8ccca29-78e6-4d63-b2b6-0f3642507a08","Type":"ContainerStarted","Data":"07145d963a352778cb6bf460cd403dd5de85fb897269eef4232095789af40b86"} Jul 08 04:18:56.997116 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.997087 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" event={"ID":"51e092a6-2a2d-4df1-8be9-81078a8081c1","Type":"ContainerStarted","Data":"a0a9d358ab413f65fa7384470234694789578057811039923fa8d21c6f8639f9"} Jul 08 04:18:56.997223 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.997150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" event={"ID":"51e092a6-2a2d-4df1-8be9-81078a8081c1","Type":"ContainerStarted","Data":"093a468fcb18ff130c8c052d1f14b4e0b5c3e50159509b3b80f76ee167d4a569"} Jul 08 04:18:56.999060 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:56.999039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzbws" event={"ID":"b5906284-7602-4e4c-b6ad-e316355875b4","Type":"ContainerStarted","Data":"9ddc73fad9e309d974c60987181d36caed7162e5f722356bb4dd0acfc4998167"} Jul 08 04:18:57.000503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.000479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" event={"ID":"6e410922-f5c2-4e4f-ac78-8a3dd1bdf9de","Type":"ContainerStarted","Data":"fa661ed7a249a1b1993b8fb4ec6a5dd059c2749df9fe6c2e4c1a7dea0f715ed3"} Jul 08 04:18:57.000665 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.000647 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:57.002221 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.002195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" event={"ID":"c5af89fe-015c-4a95-8b1a-7560c390b7ed","Type":"ContainerStarted","Data":"dc4b162b0b0d913068d1f03e0e3f98abee04ad24283ca6701678dd3c8f48fc2b"} Jul 08 04:18:57.002536 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.002515 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" Jul 08 04:18:57.003807 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.003785 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" event={"ID":"ca7f6469-7920-4c58-92be-eb8dc3791202","Type":"ContainerStarted","Data":"ae990c7c414cf240ac8dea0f7832a72fc0c788e376a5c0ee67e79bc74ff50ab8"} Jul 08 04:18:57.005860 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.005793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xcmg" event={"ID":"0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e","Type":"ContainerStarted","Data":"9d12f752e8e4d7c5db50c12ee842ed3551c5435db6e123620ee31e4e404e91c9"} Jul 08 04:18:57.012728 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.012700 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.025010 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.024962 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n7lnz" podStartSLOduration=68.7855884 podStartE2EDuration="1m22.024946438s" totalImagesPullingTime="13.239358038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.008396119 +0000 UTC m=+82.110235046" watchObservedRunningTime="2026-07-08 04:18:57.024946438 +0000 UTC m=+82.126785364" Jul 08 04:18:57.025996 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.025964 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52c44" podStartSLOduration=67.989153712 podStartE2EDuration="1m21.025952471s" totalImagesPullingTime="13.036798759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.024818624 +0000 UTC m=+82.126657571" watchObservedRunningTime="2026-07-08 04:18:57.025952471 +0000 UTC m=+82.127791377" Jul 08 04:18:57.046371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.046312 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-89vwl" podStartSLOduration=68.067048847 podStartE2EDuration="1m21.046291572s" totalImagesPullingTime="12.979242725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.045794069 +0000 UTC m=+82.147633021" watchObservedRunningTime="2026-07-08 04:18:57.046291572 +0000 UTC m=+82.148130505" Jul 08 04:18:57.065280 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.065203 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d9ff4d96-bnbz5" podStartSLOduration=1.998923698 podStartE2EDuration="14.065185485s" totalImagesPullingTime="12.066261787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.06388591 +0000 UTC m=+82.165724838" watchObservedRunningTime="2026-07-08 04:18:57.065185485 +0000 UTC m=+82.167024411" Jul 08 04:18:57.072111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.072070 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.077054 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.077028 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26z57\" (UniqueName: \"kubernetes.io/projected/08bcdf13-ab24-43df-93cb-624f8586e50e-kube-api-access-26z57\") pod \"downloads-685bfb5f64-m8q57\" (UID: \"08bcdf13-ab24-43df-93cb-624f8586e50e\") " pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:18:57.080471 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.080426 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69cd45c9df-hfcsb" podStartSLOduration=2.074678554 podStartE2EDuration="14.080410876s" totalImagesPullingTime="12.005732322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.079864828 +0000 UTC m=+82.181703754" watchObservedRunningTime="2026-07-08 04:18:57.080410876 +0000 UTC m=+82.182249803" Jul 08 04:18:57.085927 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.085905 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26z57\" (UniqueName: \"kubernetes.io/projected/08bcdf13-ab24-43df-93cb-624f8586e50e-kube-api-access-26z57\") pod \"downloads-685bfb5f64-m8q57\" (UID: \"08bcdf13-ab24-43df-93cb-624f8586e50e\") " pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:18:57.097040 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.096967 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c75b5f46d-ppskg" podStartSLOduration=1.784179093 podStartE2EDuration="14.096914643s" totalImagesPullingTime="12.31273555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.096332951 +0000 UTC m=+82.198171880" watchObservedRunningTime="2026-07-08 04:18:57.096914643 +0000 UTC m=+82.198753569" Jul 08 04:18:57.115701 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.115650 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xcmg" podStartSLOduration=34.150354542 podStartE2EDuration="47.115633805s" totalImagesPullingTime="12.965279263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.114454218 +0000 UTC m=+82.216293159" watchObservedRunningTime="2026-07-08 04:18:57.115633805 +0000 UTC m=+82.217472732" Jul 08 04:18:57.130940 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.130890 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dzbws" podStartSLOduration=71.158758974 podStartE2EDuration="1m22.130874171s" totalImagesPullingTime="10.972115197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:57.129385786 +0000 UTC m=+82.231224737" watchObservedRunningTime="2026-07-08 04:18:57.130874171 +0000 UTC m=+82.232713094" Jul 08 04:18:57.141302 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.141268 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2"] Jul 08 04:18:57.144165 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:57.144141 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff7085f_f8fa_4b10_9dc4_1b432d93cc2d.slice/crio-e8892dfc703fc363fb7ec2e1bae93fcef99b33e55b6a4e2fbebcc180985df8be WatchSource:0}: Error finding container e8892dfc703fc363fb7ec2e1bae93fcef99b33e55b6a4e2fbebcc180985df8be: Status 404 returned error can't find the container with id e8892dfc703fc363fb7ec2e1bae93fcef99b33e55b6a4e2fbebcc180985df8be Jul 08 04:18:57.243298 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.243251 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:18:57.250006 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.249960 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.304306 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.304265 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.371927 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.371905 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-m8q57"] Jul 08 04:18:57.374429 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:18:57.374401 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08bcdf13_ab24_43df_93cb_624f8586e50e.slice/crio-75c22d847f254c609ae10d0a97cfec91c1e8ddc0b07ca1d136450b23d73e7a6e WatchSource:0}: Error finding container 75c22d847f254c609ae10d0a97cfec91c1e8ddc0b07ca1d136450b23d73e7a6e: Status 404 returned error can't find the container with id 75c22d847f254c609ae10d0a97cfec91c1e8ddc0b07ca1d136450b23d73e7a6e Jul 08 04:18:57.406090 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.406062 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.458014 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.457977 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.805811 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.805753 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:57.869417 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:57.869378 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:58.010447 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.010404 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:58.013113 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.013070 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-m8q57" event={"ID":"08bcdf13-ab24-43df-93cb-624f8586e50e","Type":"ContainerStarted","Data":"75c22d847f254c609ae10d0a97cfec91c1e8ddc0b07ca1d136450b23d73e7a6e"} Jul 08 04:18:58.014538 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.014510 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" event={"ID":"bff7085f-f8fa-4b10-9dc4-1b432d93cc2d","Type":"ContainerStarted","Data":"e8892dfc703fc363fb7ec2e1bae93fcef99b33e55b6a4e2fbebcc180985df8be"} Jul 08 04:18:58.015243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.015209 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xcmg" Jul 08 04:18:58.111279 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.111145 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:58.398081 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.398047 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:58.501144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.501114 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:58.818365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:58.818325 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.015638 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.015582 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.020846 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.020811 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" event={"ID":"bff7085f-f8fa-4b10-9dc4-1b432d93cc2d","Type":"ContainerStarted","Data":"1a815cd97e4a65488092d10e0c5cbca4f24d8b5bd126425d1f2c09207a4cf1c5"} Jul 08 04:18:59.021094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.020885 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:59.026851 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.026824 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" Jul 08 04:18:59.037233 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.037184 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jtvv2" podStartSLOduration=1.828058141 podStartE2EDuration="3.037168297s" totalImagesPullingTime="1.209110156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:59.036037341 +0000 UTC m=+84.137876267" watchObservedRunningTime="2026-07-08 04:18:59.037168297 +0000 UTC m=+84.139007223" Jul 08 04:18:59.276715 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.276652 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.675330 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.675219 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.708056 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.708025 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9"] Jul 08 04:18:59.711432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.711413 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.714894 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.714791 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 04:18:59.714894 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.714798 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pnz82\"" Jul 08 04:18:59.714894 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.714795 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 04:18:59.714894 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.714837 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 04:18:59.720663 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.720630 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9"] Jul 08 04:18:59.739072 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.739049 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.806408 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.806373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.806586 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.806429 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqc22\" (UniqueName: \"kubernetes.io/projected/eaf9d1d7-882a-496e-bfa0-57f51d87020f-kube-api-access-kqc22\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.806586 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.806487 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.806586 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.806555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf9d1d7-882a-496e-bfa0-57f51d87020f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.844105 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.844021 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:18:59.906994 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.906962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.906994 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.906999 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqc22\" (UniqueName: \"kubernetes.io/projected/eaf9d1d7-882a-496e-bfa0-57f51d87020f-kube-api-access-kqc22\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.907204 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:59.907108 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 04:18:59.907204 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.907117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.907204 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:18:59.907197 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls podName:eaf9d1d7-882a-496e-bfa0-57f51d87020f nodeName:}" failed. No retries permitted until 2026-07-08 04:19:00.407174572 +0000 UTC m=+85.509013484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-bs9t9" (UID: "eaf9d1d7-882a-496e-bfa0-57f51d87020f") : secret "prometheus-operator-tls" not found Jul 08 04:18:59.907315 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.907223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf9d1d7-882a-496e-bfa0-57f51d87020f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.907922 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.907901 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaf9d1d7-882a-496e-bfa0-57f51d87020f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.909484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.909461 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:18:59.915057 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:18:59.915034 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqc22\" (UniqueName: \"kubernetes.io/projected/eaf9d1d7-882a-496e-bfa0-57f51d87020f-kube-api-access-kqc22\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:19:00.049241 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.049203 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:00.383190 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.383091 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:00.413030 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.412996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:19:00.415358 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.415326 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaf9d1d7-882a-496e-bfa0-57f51d87020f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bs9t9\" (UID: \"eaf9d1d7-882a-496e-bfa0-57f51d87020f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:19:00.484156 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.484115 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:00.621885 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.621845 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" Jul 08 04:19:00.770235 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:00.770181 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9"] Jul 08 04:19:00.772735 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:00.772705 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf9d1d7_882a_496e_bfa0_57f51d87020f.slice/crio-b4418835c5636d7b5181e692c2eb5fdbdc0c668aa15d22728a3a181cecf3633d WatchSource:0}: Error finding container b4418835c5636d7b5181e692c2eb5fdbdc0c668aa15d22728a3a181cecf3633d: Status 404 returned error can't find the container with id b4418835c5636d7b5181e692c2eb5fdbdc0c668aa15d22728a3a181cecf3633d Jul 08 04:19:01.031615 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:01.031572 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" event={"ID":"eaf9d1d7-882a-496e-bfa0-57f51d87020f","Type":"ContainerStarted","Data":"b4418835c5636d7b5181e692c2eb5fdbdc0c668aa15d22728a3a181cecf3633d"} Jul 08 04:19:02.052768 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.052715 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:02.057445 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.057416 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.061361 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061336 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 04:19:02.061361 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061359 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8fwl5\"" Jul 08 04:19:02.061553 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061391 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 04:19:02.061783 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061751 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 04:19:02.061861 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061755 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 04:19:02.061861 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.061791 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 04:19:02.065939 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.065913 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:02.130674 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.130887 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130694 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.130887 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130819 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcvcf\" (UniqueName: \"kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.130887 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.131042 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.131042 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.130988 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.231976 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.231940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcvcf\" (UniqueName: \"kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.231988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232113 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232977 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232846 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.232977 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.232928 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.233156 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.233126 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.235125 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.235079 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.235222 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.235173 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.240443 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.240417 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcvcf\" (UniqueName: \"kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf\") pod \"console-66486b595d-spj56\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.370975 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.370788 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:02.410478 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.410347 2582 patch_prober.go:28] interesting pod/image-registry-7cbfd85b89-cjqcv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 04:19:02.410478 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.410409 2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 04:19:02.565795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.565739 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:02.570796 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:02.570707 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a27af6c_5203_43b0_9fc2_b1c55bdc8108.slice/crio-194850ad3be5b6c23cae51035510b98558ab2470d5404be2c5f8ce256f06640b WatchSource:0}: Error finding container 194850ad3be5b6c23cae51035510b98558ab2470d5404be2c5f8ce256f06640b: Status 404 returned error can't find the container with id 194850ad3be5b6c23cae51035510b98558ab2470d5404be2c5f8ce256f06640b Jul 08 04:19:02.664358 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:02.664299 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:03.043919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.043876 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66486b595d-spj56" event={"ID":"0a27af6c-5203-43b0-9fc2-b1c55bdc8108","Type":"ContainerStarted","Data":"194850ad3be5b6c23cae51035510b98558ab2470d5404be2c5f8ce256f06640b"} Jul 08 04:19:03.045755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.045725 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" event={"ID":"eaf9d1d7-882a-496e-bfa0-57f51d87020f","Type":"ContainerStarted","Data":"bddd039690bdd30ab5088ac2677d3642c6f534b7186d33dbbb3ec3e411900d26"} Jul 08 04:19:03.045897 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.045791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" event={"ID":"eaf9d1d7-882a-496e-bfa0-57f51d87020f","Type":"ContainerStarted","Data":"ccc241f53cd4cfa10183f596b472ea02ff66cc32edbab68ea8b543bfa680b429"} Jul 08 04:19:03.062149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.062099 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bs9t9" podStartSLOduration=2.515081972 podStartE2EDuration="4.062081972s" totalImagesPullingTime="1.547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:03.060878379 +0000 UTC m=+88.162717327" watchObservedRunningTime="2026-07-08 04:19:03.062081972 +0000 UTC m=+88.163920903" Jul 08 04:19:03.919873 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.919824 2582 patch_prober.go:28] interesting pod/image-registry-7cbfd85b89-cjqcv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 04:19:03.920085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:03.919890 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 04:19:04.262362 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:04.262315 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:04.816362 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:04.816245 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:04.926247 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:04.926211 2582 patch_prober.go:28] interesting pod/image-registry-76d7b8c546-xx4lq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 04:19:04.926425 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:04.926271 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 04:19:05.025181 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.024120 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xcmg" Jul 08 04:19:05.043390 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.042512 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.070988 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.070909 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7"] Jul 08 04:19:05.077546 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.077524 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.080934 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.080633 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k9fxq\"" Jul 08 04:19:05.084844 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.081626 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 04:19:05.084844 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.081868 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 04:19:05.101879 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.101804 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7"] Jul 08 04:19:05.106677 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.104808 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kj5kz"] Jul 08 04:19:05.115435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.113224 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq"] Jul 08 04:19:05.116911 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.116895 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.121838 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.121586 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 04:19:05.123523 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.122062 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-grz7w\"" Jul 08 04:19:05.123523 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.122375 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 04:19:05.123523 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.122712 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 04:19:05.123967 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.123937 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq"] Jul 08 04:19:05.124284 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.124215 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.128673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.127399 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 04:19:05.128673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.127847 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 04:19:05.128673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.128093 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 04:19:05.128673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.128479 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mxtlw\"" Jul 08 04:19:05.142552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.142522 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.177732 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.177693 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-wtmp\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.178038 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-metrics-client-ca\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.178214 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxhq8\" (UniqueName: \"kubernetes.io/projected/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-kube-api-access-kxhq8\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.178379 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-sys\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.178549 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.178727 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.178947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.178931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.179147 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.179125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.179326 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.179310 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.179514 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.179500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-root\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.180038 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.180020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zsj\" (UniqueName: \"kubernetes.io/projected/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-kube-api-access-z9zsj\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.180220 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.180205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-textfile\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.180378 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.180364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.201245 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.201191 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.281966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282018 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-root\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282102 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zsj\" (UniqueName: \"kubernetes.io/projected/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-kube-api-access-z9zsj\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-textfile\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282236 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-wtmp\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-metrics-client-ca\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxhq8\" (UniqueName: \"kubernetes.io/projected/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-kube-api-access-kxhq8\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-sys\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.283301 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282466 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67f0d78f-49b2-4208-99ad-e35330cd9383-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lds8\" (UniqueName: \"kubernetes.io/projected/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-api-access-8lds8\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282571 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282594 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-sys\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.282598 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.282687 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls podName:2aa09a08-d13f-4e30-9f3d-e8f865084ccb nodeName:}" failed. No retries permitted until 2026-07-08 04:19:05.782664007 +0000 UTC m=+90.884502919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-9r8s7" (UID: "2aa09a08-d13f-4e30-9f3d-e8f865084ccb") : secret "openshift-state-metrics-tls" not found Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.282684 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-wtmp\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.283355 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.283368 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.283692 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.283754 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls podName:fa5f7f94-ba50-4db5-a2a8-c59515494a4a nodeName:}" failed. No retries permitted until 2026-07-08 04:19:05.783736444 +0000 UTC m=+90.885575355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls") pod "node-exporter-kj5kz" (UID: "fa5f7f94-ba50-4db5-a2a8-c59515494a4a") : secret "node-exporter-tls" not found Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.283821 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-root\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.284000 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-textfile\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.284462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.284283 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-metrics-client-ca\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.286784 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.286715 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.288130 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.288106 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.292276 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.292218 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zsj\" (UniqueName: \"kubernetes.io/projected/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-kube-api-access-z9zsj\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.293901 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.293855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxhq8\" (UniqueName: \"kubernetes.io/projected/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-kube-api-access-kxhq8\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384384 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67f0d78f-49b2-4208-99ad-e35330cd9383-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384445 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lds8\" (UniqueName: \"kubernetes.io/projected/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-api-access-8lds8\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.384842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.384515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.388779 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.385774 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.388779 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.386498 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67f0d78f-49b2-4208-99ad-e35330cd9383-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.388779 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.387134 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f0d78f-49b2-4208-99ad-e35330cd9383-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.403778 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.402012 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.403778 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.402367 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lds8\" (UniqueName: \"kubernetes.io/projected/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-api-access-8lds8\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.413783 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.412681 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67f0d78f-49b2-4208-99ad-e35330cd9383-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vf6kq\" (UID: \"67f0d78f-49b2-4208-99ad-e35330cd9383\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.422244 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.422143 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.445845 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.445580 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" Jul 08 04:19:05.530149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.530093 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.631233 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.631125 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:05.789326 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.789281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:05.790045 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.789968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:05.790168 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.790078 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 04:19:05.790168 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:05.790145 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls podName:fa5f7f94-ba50-4db5-a2a8-c59515494a4a nodeName:}" failed. No retries permitted until 2026-07-08 04:19:06.790124606 +0000 UTC m=+91.891963524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls") pod "node-exporter-kj5kz" (UID: "fa5f7f94-ba50-4db5-a2a8-c59515494a4a") : secret "node-exporter-tls" not found Jul 08 04:19:05.793296 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:05.793245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2aa09a08-d13f-4e30-9f3d-e8f865084ccb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9r8s7\" (UID: \"2aa09a08-d13f-4e30-9f3d-e8f865084ccb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:06.001262 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.001227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" Jul 08 04:19:06.039933 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.039787 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:06.128143 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.128112 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:19:06.128472 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.128441 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:06.134966 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.134943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.138919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.138894 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 04:19:06.139272 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.139253 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 04:19:06.139474 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.139458 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 04:19:06.140341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.139647 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 04:19:06.140341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.139903 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 04:19:06.140341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.140030 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 04:19:06.140341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.140157 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 04:19:06.140341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.140269 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 04:19:06.140655 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.140341 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 04:19:06.140655 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.140440 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-28jxz\"" Jul 08 04:19:06.149363 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.149244 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:19:06.237553 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.237477 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:06.297254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.296912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.297254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.296959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.297254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.296985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n749g\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.297254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.297042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.297254 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.297070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.297093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298601 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.298746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.298633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.359799 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.358685 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:06.399935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.399900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.400208 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.400187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.400382 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.400364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n749g\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.400633 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.400615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.400808 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.400792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.402018 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.401998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.402596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.402578 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.402775 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.402743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.402954 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.402935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.414613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.403442 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.415339 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.415253 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.415503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.401919 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.415503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.401177 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.415985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.415948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.416433 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.416372 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.416542 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.416530 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.418164 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n749g\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.418226 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.419838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.419932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.419962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.421094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.419997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.431341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.431265 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.431463 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.431395 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.431744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.431720 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.432689 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.432552 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.457633 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.457552 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:19:06.506455 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.506411 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq"] Jul 08 04:19:06.511386 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:06.511357 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f0d78f_49b2_4208_99ad_e35330cd9383.slice/crio-1687416c05daa0c4acf74b11912c3736b753bb7bb27bd973580ae1c358996f61 WatchSource:0}: Error finding container 1687416c05daa0c4acf74b11912c3736b753bb7bb27bd973580ae1c358996f61: Status 404 returned error can't find the container with id 1687416c05daa0c4acf74b11912c3736b753bb7bb27bd973580ae1c358996f61 Jul 08 04:19:06.536047 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.535625 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7"] Jul 08 04:19:06.673293 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.673262 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:19:06.676185 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:06.676144 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b4296d_c54b_40b7_b3f0_d7ef6011bca8.slice/crio-74376edcf8bb70f248cbf7c66093b05bb146b6d270a8b42ec56cda6b7d3b9ab5 WatchSource:0}: Error finding container 74376edcf8bb70f248cbf7c66093b05bb146b6d270a8b42ec56cda6b7d3b9ab5: Status 404 returned error can't find the container with id 74376edcf8bb70f248cbf7c66093b05bb146b6d270a8b42ec56cda6b7d3b9ab5 Jul 08 04:19:06.825702 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.825523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:06.828577 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.828541 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa5f7f94-ba50-4db5-a2a8-c59515494a4a-node-exporter-tls\") pod \"node-exporter-kj5kz\" (UID: \"fa5f7f94-ba50-4db5-a2a8-c59515494a4a\") " pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:06.935994 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:06.935956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kj5kz" Jul 08 04:19:07.084327 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.084107 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" event={"ID":"2aa09a08-d13f-4e30-9f3d-e8f865084ccb","Type":"ContainerStarted","Data":"22107acf56bd19974fd579bc72887d96ee3a19285475e4c1ec77d0b6f19c766f"} Jul 08 04:19:07.084327 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.084187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" event={"ID":"2aa09a08-d13f-4e30-9f3d-e8f865084ccb","Type":"ContainerStarted","Data":"40b930c48e9874c62efbcc471c491c6c9bc34f6732df1bc07535b177fb7245d5"} Jul 08 04:19:07.084327 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.084218 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" event={"ID":"2aa09a08-d13f-4e30-9f3d-e8f865084ccb","Type":"ContainerStarted","Data":"02d9007ee6bf5a2f0d0639dfcc1349315f9033b34d396573c93741de383bcfa8"} Jul 08 04:19:07.086902 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.086829 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66486b595d-spj56" event={"ID":"0a27af6c-5203-43b0-9fc2-b1c55bdc8108","Type":"ContainerStarted","Data":"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b"} Jul 08 04:19:07.089259 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.089227 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"74376edcf8bb70f248cbf7c66093b05bb146b6d270a8b42ec56cda6b7d3b9ab5"} Jul 08 04:19:07.090899 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.090871 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kj5kz" event={"ID":"fa5f7f94-ba50-4db5-a2a8-c59515494a4a","Type":"ContainerStarted","Data":"8895678a7717fe72a8f0cb7fe5799586e1287c233b483885c72058b27e5a4d46"} Jul 08 04:19:07.092352 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.092325 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" event={"ID":"67f0d78f-49b2-4208-99ad-e35330cd9383","Type":"ContainerStarted","Data":"1687416c05daa0c4acf74b11912c3736b753bb7bb27bd973580ae1c358996f61"} Jul 08 04:19:07.103359 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.103301 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66486b595d-spj56" podStartSLOduration=1.30666066 podStartE2EDuration="5.103288317s" totalImagesPullingTime="3.796627657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:07.102943872 +0000 UTC m=+92.204782823" watchObservedRunningTime="2026-07-08 04:19:07.103288317 +0000 UTC m=+92.205127243" Jul 08 04:19:07.978220 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:07.978125 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:08.055557 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.055522 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d4c57d75b-jxs8n"] Jul 08 04:19:08.056979 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.056940 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:08.061668 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.061647 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.064312 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.064204 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 04:19:08.065579 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.064400 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 04:19:08.065579 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.064802 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9f8x6\"" Jul 08 04:19:08.078263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.066319 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 04:19:08.078263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.066525 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 04:19:08.078263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.066709 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-rdmaf21klven\"" Jul 08 04:19:08.078263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.074868 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 04:19:08.078263 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.075128 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d4c57d75b-jxs8n"] Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240600 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2s76\" (UniqueName: \"kubernetes.io/projected/bef8e0c6-74bc-4247-b5ff-793e373d31f1-kube-api-access-g2s76\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bef8e0c6-74bc-4247-b5ff-793e373d31f1-metrics-client-ca\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241380 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.240963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-grpc-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.241936 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.241808 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:08.341812 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.341740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.342239 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342201 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:08.342439 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-grpc-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2s76\" (UniqueName: \"kubernetes.io/projected/bef8e0c6-74bc-4247-b5ff-793e373d31f1-kube-api-access-g2s76\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.342750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bef8e0c6-74bc-4247-b5ff-793e373d31f1-metrics-client-ca\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.343755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.343600 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bef8e0c6-74bc-4247-b5ff-793e373d31f1-metrics-client-ca\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.347212 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.347157 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.349228 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.349171 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.349332 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.349303 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.350021 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.349990 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.350390 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.350365 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.351292 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.351253 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bef8e0c6-74bc-4247-b5ff-793e373d31f1-secret-grpc-tls\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.351386 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.351330 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2s76\" (UniqueName: \"kubernetes.io/projected/bef8e0c6-74bc-4247-b5ff-793e373d31f1-kube-api-access-g2s76\") pod \"thanos-querier-d4c57d75b-jxs8n\" (UID: \"bef8e0c6-74bc-4247-b5ff-793e373d31f1\") " pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.383164 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.383133 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:08.395975 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.395912 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:08.629040 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:08.628996 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:09.147234 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.146743 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d4c57d75b-jxs8n"] Jul 08 04:19:09.152883 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:09.152849 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef8e0c6_74bc_4247_b5ff_793e373d31f1.slice/crio-ca1133f405e9030883c499bf8b51683969e6cd8e25b9d3f48a6cc57a16144d84 WatchSource:0}: Error finding container ca1133f405e9030883c499bf8b51683969e6cd8e25b9d3f48a6cc57a16144d84: Status 404 returned error can't find the container with id ca1133f405e9030883c499bf8b51683969e6cd8e25b9d3f48a6cc57a16144d84 Jul 08 04:19:09.474599 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.474511 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5978c597bb-vxd52"] Jul 08 04:19:09.478700 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.478667 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.480978 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.480954 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 04:19:09.481205 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.481181 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 04:19:09.481299 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.481288 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5jo1gqfafqoa1\"" Jul 08 04:19:09.481365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.481314 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 04:19:09.481365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.481349 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 04:19:09.481943 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.481929 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xrfnh\"" Jul 08 04:19:09.484733 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.484697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5978c597bb-vxd52"] Jul 08 04:19:09.555874 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.555841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-client-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556084 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.555883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4psx\" (UniqueName: \"kubernetes.io/projected/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-kube-api-access-d4psx\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556084 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.555935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-client-certs\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556084 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.555985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-audit-log\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556084 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.556054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-metrics-server-audit-profiles\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556293 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.556183 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-tls\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.556293 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.556241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.657704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.657665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-metrics-server-audit-profiles\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-tls\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658186 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658329 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-client-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658429 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4psx\" (UniqueName: \"kubernetes.io/projected/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-kube-api-access-d4psx\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-client-certs\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658490 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-audit-log\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.658896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658868 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-audit-log\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.659002 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.658936 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.659076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.659002 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-metrics-server-audit-profiles\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.661502 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.661481 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-client-ca-bundle\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.661600 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.661485 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-client-certs\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.661892 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.661865 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-secret-metrics-server-tls\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.666866 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.666837 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4psx\" (UniqueName: \"kubernetes.io/projected/bf9cabc5-d012-492f-bef9-dc36c8f66c3d-kube-api-access-d4psx\") pod \"metrics-server-5978c597bb-vxd52\" (UID: \"bf9cabc5-d012-492f-bef9-dc36c8f66c3d\") " pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.790526 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.790467 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:09.850801 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.850691 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:09.858850 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.858076 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw"] Jul 08 04:19:09.863893 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.863829 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:09.868256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.868044 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-82bmq\"" Jul 08 04:19:09.868256 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.868122 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 04:19:09.882774 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.881529 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw"] Jul 08 04:19:09.944435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.944201 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:09.951314 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.951076 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerName="registry" containerID="cri-o://9a542bc5c2097441710ce179e868888ede286b3da71a13f36444c8be8cb7e00b" gracePeriod=30 Jul 08 04:19:09.962085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.962053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sksq6\" (UniqueName: \"kubernetes.io/projected/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-kube-api-access-sksq6\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:09.962221 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.962120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:09.995034 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:09.994946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5978c597bb-vxd52"] Jul 08 04:19:10.001532 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:10.001441 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9cabc5_d012_492f_bef9_dc36c8f66c3d.slice/crio-071c31ce350fb860091d34a8dfb8b888bfcefa3851512b2967a511b7231256a3 WatchSource:0}: Error finding container 071c31ce350fb860091d34a8dfb8b888bfcefa3851512b2967a511b7231256a3: Status 404 returned error can't find the container with id 071c31ce350fb860091d34a8dfb8b888bfcefa3851512b2967a511b7231256a3 Jul 08 04:19:10.063584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.063240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sksq6\" (UniqueName: \"kubernetes.io/projected/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-kube-api-access-sksq6\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:10.063584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.063290 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:10.063584 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:10.063504 2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 04:19:10.063584 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:10.063561 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert podName:c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:10.563541007 +0000 UTC m=+95.665379916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-ffvxw" (UID: "c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6") : secret "monitoring-plugin-cert" not found Jul 08 04:19:10.072121 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.072057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sksq6\" (UniqueName: \"kubernetes.io/projected/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-kube-api-access-sksq6\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:10.118754 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.118714 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerID="9a542bc5c2097441710ce179e868888ede286b3da71a13f36444c8be8cb7e00b" exitCode=0 Jul 08 04:19:10.118997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.118859 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" event={"ID":"ec857cc2-e1ed-4045-be8e-93a05523ab8e","Type":"ContainerDied","Data":"9a542bc5c2097441710ce179e868888ede286b3da71a13f36444c8be8cb7e00b"} Jul 08 04:19:10.121157 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.121126 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0" exitCode=0 Jul 08 04:19:10.121269 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.121211 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0"} Jul 08 04:19:10.123691 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.123512 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa5f7f94-ba50-4db5-a2a8-c59515494a4a" containerID="1ffc9aea8338aeeff98af52964c293b64619eb7686411dcbb3c7eb4842031beb" exitCode=0 Jul 08 04:19:10.123691 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.123650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kj5kz" event={"ID":"fa5f7f94-ba50-4db5-a2a8-c59515494a4a","Type":"ContainerDied","Data":"1ffc9aea8338aeeff98af52964c293b64619eb7686411dcbb3c7eb4842031beb"} Jul 08 04:19:10.128370 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.128343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" event={"ID":"67f0d78f-49b2-4208-99ad-e35330cd9383","Type":"ContainerStarted","Data":"89df484698fd658e3046047f5ccf3ae2fec5f8fdef03a02589ce76dde2e4b295"} Jul 08 04:19:10.128459 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.128400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" event={"ID":"67f0d78f-49b2-4208-99ad-e35330cd9383","Type":"ContainerStarted","Data":"48dba070aeb0ab8b55fb6cb56d2b8d2db7bacabda714571a5ff60fffd88b454e"} Jul 08 04:19:10.128459 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.128427 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" event={"ID":"67f0d78f-49b2-4208-99ad-e35330cd9383","Type":"ContainerStarted","Data":"11dd111a49d31d9a788af8bfb57ffb0f79b8da463f5fcff759540f226e5e069d"} Jul 08 04:19:10.134799 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.134746 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" event={"ID":"2aa09a08-d13f-4e30-9f3d-e8f865084ccb","Type":"ContainerStarted","Data":"b5b8194f5f04c0a6d7f3a56edd6298c69a539305a9252c9466d77edba7fd1252"} Jul 08 04:19:10.137300 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.137267 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" event={"ID":"bf9cabc5-d012-492f-bef9-dc36c8f66c3d","Type":"ContainerStarted","Data":"071c31ce350fb860091d34a8dfb8b888bfcefa3851512b2967a511b7231256a3"} Jul 08 04:19:10.139398 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.139373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"ca1133f405e9030883c499bf8b51683969e6cd8e25b9d3f48a6cc57a16144d84"} Jul 08 04:19:10.167887 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.167809 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9r8s7" podStartSLOduration=2.926205592 podStartE2EDuration="5.167784979s" totalImagesPullingTime="2.241579387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:10.167646107 +0000 UTC m=+95.269485065" watchObservedRunningTime="2026-07-08 04:19:10.167784979 +0000 UTC m=+95.269623910" Jul 08 04:19:10.213225 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.213152 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vf6kq" podStartSLOduration=2.707280274 podStartE2EDuration="5.213123506s" totalImagesPullingTime="2.505843232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:10.207561509 +0000 UTC m=+95.309400439" watchObservedRunningTime="2026-07-08 04:19:10.213123506 +0000 UTC m=+95.314962435" Jul 08 04:19:10.225441 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.225326 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:19:10.237225 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.236260 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:10.267134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.266879 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.266939 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267114 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267346 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267164 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267346 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267198 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267497 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267472 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndwn9\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267585 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267542 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.267641 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.267587 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates\") pod \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\" (UID: \"ec857cc2-e1ed-4045-be8e-93a05523ab8e\") " Jul 08 04:19:10.268526 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.268354 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:10.268788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.268617 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:10.271432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.271405 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:10.271896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.271870 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:10.272214 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.272170 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9" (OuterVolumeSpecName: "kube-api-access-ndwn9") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "kube-api-access-ndwn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:10.272382 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.272350 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:10.273641 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.273602 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:10.282149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.282061 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ec857cc2-e1ed-4045-be8e-93a05523ab8e" (UID: "ec857cc2-e1ed-4045-be8e-93a05523ab8e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:19:10.369750 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369715 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-bound-sa-token\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369751 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-trusted-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369793 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369806 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-installation-pull-secrets\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369822 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec857cc2-e1ed-4045-be8e-93a05523ab8e-ca-trust-extracted\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369835 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndwn9\" (UniqueName: \"kubernetes.io/projected/ec857cc2-e1ed-4045-be8e-93a05523ab8e-kube-api-access-ndwn9\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369848 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec857cc2-e1ed-4045-be8e-93a05523ab8e-image-registry-private-configuration\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.369915 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.369862 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec857cc2-e1ed-4045-be8e-93a05523ab8e-registry-certificates\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:10.439312 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.439260 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:10.450878 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.450833 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:19:10.451454 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.451429 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerName="registry" Jul 08 04:19:10.451454 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.451453 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerName="registry" Jul 08 04:19:10.451697 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.451554 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" containerName="registry" Jul 08 04:19:10.455288 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.455263 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.463853 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.463826 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:19:10.464020 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.464002 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.470895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.470962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.471026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.471051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mjf5\" (UniqueName: \"kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.471088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.471200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.471246 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.471224 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.536491 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.536446 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:10.572729 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.572688 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.572919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.572755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:10.572919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.572817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.572919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.572842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mjf5\" (UniqueName: \"kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.572919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.572894 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.573150 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.573030 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.573150 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.573057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.573150 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.573085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.573577 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.573525 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.574058 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.574018 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.574261 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.574242 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.574556 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.574533 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.576629 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.576605 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.576829 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.576727 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ffvxw\" (UID: \"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:10.576993 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.576964 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.581577 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.581551 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mjf5\" (UniqueName: \"kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5\") pod \"console-84bdc9c4cd-9rlmw\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.772423 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.772386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:10.784134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:10.783881 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:11.134279 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.134111 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:11.151784 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.150847 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" event={"ID":"ec857cc2-e1ed-4045-be8e-93a05523ab8e","Type":"ContainerDied","Data":"4be81ae5118fdf8f820ba76c35973fcc9eb0252da2292d206a55161fc0b22f74"} Jul 08 04:19:11.151784 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.150939 2582 scope.go:117] "RemoveContainer" containerID="9a542bc5c2097441710ce179e868888ede286b3da71a13f36444c8be8cb7e00b" Jul 08 04:19:11.153789 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.152154 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d7b8c546-xx4lq" Jul 08 04:19:11.158151 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.157811 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kj5kz" event={"ID":"fa5f7f94-ba50-4db5-a2a8-c59515494a4a","Type":"ContainerStarted","Data":"5bfc0d045e3d9407f6006220d0fd4d645423978c77c007cb0fafc7299916646c"} Jul 08 04:19:11.158151 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.157859 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kj5kz" event={"ID":"fa5f7f94-ba50-4db5-a2a8-c59515494a4a","Type":"ContainerStarted","Data":"a49c4c84415f6b16d87c6667fdf6b5fa578844e06143fd72ef251df6eb3127d4"} Jul 08 04:19:11.184460 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.184401 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kj5kz" podStartSLOduration=4.075190282 podStartE2EDuration="6.184378741s" totalImagesPullingTime="2.013418776s" totalInitContainerRuntime="95.769683ms" isStatefulPod=true podCreationTimestamp="2026-07-08 04:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:11.180880233 +0000 UTC m=+96.282719176" watchObservedRunningTime="2026-07-08 04:19:11.184378741 +0000 UTC m=+96.286217654" Jul 08 04:19:11.197419 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.197330 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:19:11.201083 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.201024 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76d7b8c546-xx4lq"] Jul 08 04:19:11.305555 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.305402 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" limit=1024 Jul 08 04:19:11.306300 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.306065 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:19:11.314408 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.314386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.317125 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.317091 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 04:19:11.318046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.317587 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 04:19:11.318046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.317667 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 04:19:11.318046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.317601 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 04:19:11.318241 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318105 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 04:19:11.318293 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318270 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 04:19:11.318357 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318308 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zdthz\"" Jul 08 04:19:11.318572 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318552 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 04:19:11.318897 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318875 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 04:19:11.319000 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.318935 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 04:19:11.319203 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.319187 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 04:19:11.319744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.319730 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 04:19:11.320092 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.320067 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ab09pqc7mv9e6\"" Jul 08 04:19:11.320909 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.320888 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 04:19:11.322808 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.322782 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 04:19:11.326273 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.326212 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:19:11.326404 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.326382 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 04:19:11.383458 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383458 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383458 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383389 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383458 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383557 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383627 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383668 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383702 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt7hr\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383742 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.383803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.383973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.384018 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.384060 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.384105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384473 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.384142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.384473 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.384202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.452124 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.452070 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec857cc2-e1ed-4045-be8e-93a05523ab8e" path="/var/lib/kubelet/pods/ec857cc2-e1ed-4045-be8e-93a05523ab8e/volumes" Jul 08 04:19:11.485164 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485131 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485164 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485209 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485369 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485580 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485728 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt7hr\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.485863 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.485974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.486016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.486057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486437 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.486333 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.486502 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.486464 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.487814 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.486623 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.487814 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.487049 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.487814 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.487193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.488407 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.488338 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.488775 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.488719 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.489952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.489897 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.490231 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.490204 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.490991 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.490911 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.490991 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.490937 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.491238 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.491213 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.491403 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.491384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.492149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.491498 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.492149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.491633 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.492149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.492065 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.492741 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.492718 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.493056 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.493033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.494916 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.494875 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt7hr\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr\") pod \"prometheus-k8s-0\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:11.631581 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:11.631531 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:12.371474 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:12.371436 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:12.371978 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:12.371801 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:12.377261 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:12.377238 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:12.410008 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:12.409961 2582 patch_prober.go:28] interesting pod/image-registry-7cbfd85b89-cjqcv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 04:19:12.410171 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:12.410036 2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 04:19:13.173682 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:13.173652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:13.919923 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:13.919886 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:19:14.923417 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:14.923388 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vjp7n" Jul 08 04:19:17.011181 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:17.011132 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b8f8b8cb4-9dg6n" Jul 08 04:19:17.061552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:17.061181 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:19:20.759550 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:20.759475 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:19:20.761233 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:20.761199 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6523611c_7bf5_41f0_a21b_6127aac61cc7.slice/crio-1656ee92ec387eb9a28108e43649fdc17bed76940970dccfb91a489f4316cd78 WatchSource:0}: Error finding container 1656ee92ec387eb9a28108e43649fdc17bed76940970dccfb91a489f4316cd78: Status 404 returned error can't find the container with id 1656ee92ec387eb9a28108e43649fdc17bed76940970dccfb91a489f4316cd78 Jul 08 04:19:20.977976 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:20.977946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw"] Jul 08 04:19:20.979955 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:19:20.979927 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19d9eda_ad8a_4d28_ac91_5594d1b0f7a6.slice/crio-1df5e50a4bce737dcf84bef4dd7be07803e760a81a67429da8776029776fba7f WatchSource:0}: Error finding container 1df5e50a4bce737dcf84bef4dd7be07803e760a81a67429da8776029776fba7f: Status 404 returned error can't find the container with id 1df5e50a4bce737dcf84bef4dd7be07803e760a81a67429da8776029776fba7f Jul 08 04:19:20.992132 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:20.992093 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:19:21.217394 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.217305 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bdc9c4cd-9rlmw" event={"ID":"6523611c-7bf5-41f0-a21b-6127aac61cc7","Type":"ContainerStarted","Data":"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c"} Jul 08 04:19:21.217394 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.217357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bdc9c4cd-9rlmw" event={"ID":"6523611c-7bf5-41f0-a21b-6127aac61cc7","Type":"ContainerStarted","Data":"1656ee92ec387eb9a28108e43649fdc17bed76940970dccfb91a489f4316cd78"} Jul 08 04:19:21.219960 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.219658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-m8q57" event={"ID":"08bcdf13-ab24-43df-93cb-624f8586e50e","Type":"ContainerStarted","Data":"80619d270d12f996b26f3c186bf309e09dc6a609ca8bfb0ed8beadc17159085a"} Jul 08 04:19:21.220519 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.220495 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:19:21.223347 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.223222 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" event={"ID":"bf9cabc5-d012-492f-bef9-dc36c8f66c3d","Type":"ContainerStarted","Data":"041fb5a9ea8f8d0dbe053f28bc7f24c4dd7cd007096a54b0152e3fab448ca85f"} Jul 08 04:19:21.224810 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.224787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" event={"ID":"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6","Type":"ContainerStarted","Data":"1df5e50a4bce737dcf84bef4dd7be07803e760a81a67429da8776029776fba7f"} Jul 08 04:19:21.227671 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.227642 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"e243ad3c9b53354b4d8fea4d3122b105708076503509ea2ec3363719144b6e86"} Jul 08 04:19:21.227806 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.227718 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"95a0239f601661c515f2bef4740dacb4ebce7a4e0b550302694db0f33d8d583a"} Jul 08 04:19:21.227806 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.227785 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"d4cd35cb1e8ec962ac4f179b06f192a645dd81d84bccb8047dc34827252adebb"} Jul 08 04:19:21.230482 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.230463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-m8q57" Jul 08 04:19:21.231839 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.231812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73"} Jul 08 04:19:21.231926 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.231886 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9"} Jul 08 04:19:21.231995 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.231928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487"} Jul 08 04:19:21.231995 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.231973 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e"} Jul 08 04:19:21.234320 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.234299 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb" exitCode=0 Jul 08 04:19:21.234574 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.234455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb"} Jul 08 04:19:21.234574 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.234522 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"d8c2ef4e736c3b6b41c51905e5ee683fb7bb00243c9a56904a2cc7f3bf4343ee"} Jul 08 04:19:21.243460 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.243349 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84bdc9c4cd-9rlmw" podStartSLOduration=11.243317507 podStartE2EDuration="11.243317507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:21.242514068 +0000 UTC m=+106.344352994" watchObservedRunningTime="2026-07-08 04:19:21.243317507 +0000 UTC m=+106.345156436" Jul 08 04:19:21.264275 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.264183 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-m8q57" podStartSLOduration=2.098161162 podStartE2EDuration="25.264167383s" totalImagesPullingTime="23.166006221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:21.261141605 +0000 UTC m=+106.362980567" watchObservedRunningTime="2026-07-08 04:19:21.264167383 +0000 UTC m=+106.366006336" Jul 08 04:19:21.278618 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:21.278562 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" podStartSLOduration=1.797023223 podStartE2EDuration="12.278539051s" totalImagesPullingTime="10.481515828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:21.277194872 +0000 UTC m=+106.379033806" watchObservedRunningTime="2026-07-08 04:19:21.278539051 +0000 UTC m=+106.380377980" Jul 08 04:19:22.245847 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:22.245806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702"} Jul 08 04:19:24.263953 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.263874 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" event={"ID":"c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6","Type":"ContainerStarted","Data":"ad8a05987c75ec3d2cbe39e20a7381bd9362750be993334da339d38504dccb1b"} Jul 08 04:19:24.264552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.264524 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:24.269475 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.269393 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"9f2ea2fbe718b02eac2188ceadde7fa4c6c85cd8252b85b3a37482bc560844d9"} Jul 08 04:19:24.269645 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.269482 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:24.269645 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.269529 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"585678fb507520c943b1cacd27bcdb77e6b5bce3d46a3b493d4ca7850afc8aed"} Jul 08 04:19:24.269645 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.269575 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" event={"ID":"bef8e0c6-74bc-4247-b5ff-793e373d31f1","Type":"ContainerStarted","Data":"222e3ff8a97620933cab686de55b4d9ae8d6eb244e66094e8344d0b070914b98"} Jul 08 04:19:24.273198 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.273130 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" Jul 08 04:19:24.276630 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.276531 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerStarted","Data":"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c"} Jul 08 04:19:24.282896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.282825 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ffvxw" podStartSLOduration=12.836988062 podStartE2EDuration="15.282806649s" totalImagesPullingTime="2.445818587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:24.280653603 +0000 UTC m=+109.382492568" watchObservedRunningTime="2026-07-08 04:19:24.282806649 +0000 UTC m=+109.384645580" Jul 08 04:19:24.303434 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.303372 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" podStartSLOduration=2.484095464 podStartE2EDuration="16.303349361s" totalImagesPullingTime="13.819253897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:24.302211276 +0000 UTC m=+109.404050254" watchObservedRunningTime="2026-07-08 04:19:24.303349361 +0000 UTC m=+109.405188291" Jul 08 04:19:24.338891 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:24.338827 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.4700264499999998 podStartE2EDuration="18.338803819s" totalImagesPullingTime="14.837978868s" totalInitContainerRuntime="30.798501ms" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:24.335057528 +0000 UTC m=+109.436896494" watchObservedRunningTime="2026-07-08 04:19:24.338803819 +0000 UTC m=+109.440642752" Jul 08 04:19:25.291400 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:25.291370 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d4c57d75b-jxs8n" Jul 08 04:19:28.318488 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318441 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3"} Jul 08 04:19:28.318997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c"} Jul 08 04:19:28.318997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318605 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3"} Jul 08 04:19:28.318997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85"} Jul 08 04:19:28.318997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341"} Jul 08 04:19:28.318997 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.318715 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerStarted","Data":"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1"} Jul 08 04:19:28.350664 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:28.350599 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=11.385791048 podStartE2EDuration="17.350571403s" totalImagesPullingTime="5.938478008s" totalInitContainerRuntime="26.302347ms" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:28.346790063 +0000 UTC m=+113.448629003" watchObservedRunningTime="2026-07-08 04:19:28.350571403 +0000 UTC m=+113.452410329" Jul 08 04:19:29.791065 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:29.791019 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:29.791653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:29.791108 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:30.772822 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:30.772754 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:30.772822 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:30.772828 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:30.777866 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:30.777841 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:31.338570 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:31.338535 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:19:31.384039 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:31.383994 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:31.631955 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:31.631862 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:19:42.090434 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.090361 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" containerID="cri-o://33b20395acea1b0c9b42768b3aad35806a4f67ed32cc966ced8f64bb12603730" gracePeriod=30 Jul 08 04:19:42.394857 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.394820 2582 generic.go:356] "Generic (PLEG): container finished" podID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerID="33b20395acea1b0c9b42768b3aad35806a4f67ed32cc966ced8f64bb12603730" exitCode=0 Jul 08 04:19:42.395012 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.394893 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" event={"ID":"de847b82-b567-41b9-9f9e-92ee2815ca85","Type":"ContainerDied","Data":"33b20395acea1b0c9b42768b3aad35806a4f67ed32cc966ced8f64bb12603730"} Jul 08 04:19:42.395012 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.394948 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" event={"ID":"de847b82-b567-41b9-9f9e-92ee2815ca85","Type":"ContainerDied","Data":"c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934"} Jul 08 04:19:42.395012 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.394969 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4daf1978a6f59cd5ef28e8467bf75e03c6435b287f61b82aa017b88f952a934" Jul 08 04:19:42.402597 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.402570 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:19:42.437104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437076 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqltg\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437140 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437162 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437181 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437209 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437251 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437562 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437321 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437562 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437369 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates\") pod \"de847b82-b567-41b9-9f9e-92ee2815ca85\" (UID: \"de847b82-b567-41b9-9f9e-92ee2815ca85\") " Jul 08 04:19:42.437902 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.437868 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:42.438456 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.438402 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:42.440679 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.440652 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg" (OuterVolumeSpecName: "kube-api-access-tqltg") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "kube-api-access-tqltg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:42.440803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.440732 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:42.440876 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.440860 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:42.440955 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.440929 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:42.441179 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.441156 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:42.446793 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.446634 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "de847b82-b567-41b9-9f9e-92ee2815ca85" (UID: "de847b82-b567-41b9-9f9e-92ee2815ca85"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538435 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-image-registry-private-configuration\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538474 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de847b82-b567-41b9-9f9e-92ee2815ca85-installation-pull-secrets\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538490 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538504 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-trusted-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538516 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de847b82-b567-41b9-9f9e-92ee2815ca85-ca-trust-extracted\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538529 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-bound-sa-token\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538543 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de847b82-b567-41b9-9f9e-92ee2815ca85-registry-certificates\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:42.538604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:42.538555 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqltg\" (UniqueName: \"kubernetes.io/projected/de847b82-b567-41b9-9f9e-92ee2815ca85-kube-api-access-tqltg\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:43.399833 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:43.399795 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbfd85b89-cjqcv" Jul 08 04:19:43.421008 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:43.420972 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:19:43.424446 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:43.424405 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cbfd85b89-cjqcv"] Jul 08 04:19:43.447028 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:43.446995 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" path="/var/lib/kubelet/pods/de847b82-b567-41b9-9f9e-92ee2815ca85/volumes" Jul 08 04:19:46.416575 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:46.416537 2582 generic.go:356] "Generic (PLEG): container finished" podID="63781dc0-d73d-4209-9cd7-49951c8693eb" containerID="c9e4efec9d56093fdef50d8cf8624fb7e08dbc9e48238af2513f69fef87cc123" exitCode=0 Jul 08 04:19:46.417034 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:46.416623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" event={"ID":"63781dc0-d73d-4209-9cd7-49951c8693eb","Type":"ContainerDied","Data":"c9e4efec9d56093fdef50d8cf8624fb7e08dbc9e48238af2513f69fef87cc123"} Jul 08 04:19:46.417034 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:46.417003 2582 scope.go:117] "RemoveContainer" containerID="c9e4efec9d56093fdef50d8cf8624fb7e08dbc9e48238af2513f69fef87cc123" Jul 08 04:19:47.423809 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:47.423751 2582 generic.go:356] "Generic (PLEG): container finished" podID="10be44bf-2a06-440b-baf5-951568059b8c" containerID="b441e8f8ae6a9c7fdcd4400da5629a2cb596d4ccf4dbd19c8726b82f1d230d15" exitCode=0 Jul 08 04:19:47.424314 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:47.423821 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" event={"ID":"10be44bf-2a06-440b-baf5-951568059b8c","Type":"ContainerDied","Data":"b441e8f8ae6a9c7fdcd4400da5629a2cb596d4ccf4dbd19c8726b82f1d230d15"} Jul 08 04:19:47.424314 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:47.424229 2582 scope.go:117] "RemoveContainer" containerID="b441e8f8ae6a9c7fdcd4400da5629a2cb596d4ccf4dbd19c8726b82f1d230d15" Jul 08 04:19:47.426040 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:47.426020 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-875fj" event={"ID":"63781dc0-d73d-4209-9cd7-49951c8693eb","Type":"ContainerStarted","Data":"b04d908db190beae56fb4a68bd075b24824489691ee2154187101b674314b31c"} Jul 08 04:19:48.432737 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:48.432702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vrrtx" event={"ID":"10be44bf-2a06-440b-baf5-951568059b8c","Type":"ContainerStarted","Data":"11aee34c6961078c7a40f2a5f33638f1c89046327c0ee9c1569acefb08df98ca"} Jul 08 04:19:49.797018 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:49.796987 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:49.800902 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:49.800875 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5978c597bb-vxd52" Jul 08 04:19:51.450221 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:51.450139 2582 generic.go:356] "Generic (PLEG): container finished" podID="4ecfabab-c5bf-425e-9d4e-a74550dd2754" containerID="129936a5711d7939ab80fa601f7fa75102e1a35f623e833063af99dbeb5f88ca" exitCode=0 Jul 08 04:19:51.450221 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:51.450206 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" event={"ID":"4ecfabab-c5bf-425e-9d4e-a74550dd2754","Type":"ContainerDied","Data":"129936a5711d7939ab80fa601f7fa75102e1a35f623e833063af99dbeb5f88ca"} Jul 08 04:19:51.450644 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:51.450553 2582 scope.go:117] "RemoveContainer" containerID="129936a5711d7939ab80fa601f7fa75102e1a35f623e833063af99dbeb5f88ca" Jul 08 04:19:52.441869 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:52.441845 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdfc5556f-2c48c_e0fd4931-ebcc-4948-b5fd-748844c3c538/router/0.log" Jul 08 04:19:52.457483 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:52.457453 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nl96k" event={"ID":"4ecfabab-c5bf-425e-9d4e-a74550dd2754","Type":"ContainerStarted","Data":"ba9113bc4c7f5a845e686b45bd84b00d6b7bcb85cf4f9d4add878d8db31ad526"} Jul 08 04:19:52.469721 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:52.469693 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sdfj6_7d1da944-0214-4cd8-87be-838ee702e3ca/serve-healthcheck-canary/0.log" Jul 08 04:19:56.407535 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.407472 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66486b595d-spj56" podUID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" containerName="console" containerID="cri-o://0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b" gracePeriod=15 Jul 08 04:19:56.708237 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.708205 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:56.783228 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783196 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783420 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783252 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783420 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783291 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783420 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783379 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcvcf\" (UniqueName: \"kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783579 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783440 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783579 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783491 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca\") pod \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\" (UID: \"0a27af6c-5203-43b0-9fc2-b1c55bdc8108\") " Jul 08 04:19:56.783869 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783809 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:56.784003 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.783899 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config" (OuterVolumeSpecName: "console-config") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:56.784068 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.784041 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:19:56.785860 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.785841 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:56.785969 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.785951 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf" (OuterVolumeSpecName: "kube-api-access-hcvcf") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "kube-api-access-hcvcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:19:56.786074 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.785950 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a27af6c-5203-43b0-9fc2-b1c55bdc8108" (UID: "0a27af6c-5203-43b0-9fc2-b1c55bdc8108"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:19:56.884852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884815 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-oauth-serving-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:56.884852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884845 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-serving-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:56.884852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884856 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-oauth-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:56.885085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884865 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcvcf\" (UniqueName: \"kubernetes.io/projected/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-kube-api-access-hcvcf\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:56.885085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884876 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-console-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:56.885085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:56.884885 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a27af6c-5203-43b0-9fc2-b1c55bdc8108-service-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:19:57.483931 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.483896 2582 generic.go:356] "Generic (PLEG): container finished" podID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" containerID="0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b" exitCode=0 Jul 08 04:19:57.484432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.483955 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66486b595d-spj56" event={"ID":"0a27af6c-5203-43b0-9fc2-b1c55bdc8108","Type":"ContainerDied","Data":"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b"} Jul 08 04:19:57.484432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.483969 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66486b595d-spj56" Jul 08 04:19:57.484432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.484011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66486b595d-spj56" event={"ID":"0a27af6c-5203-43b0-9fc2-b1c55bdc8108","Type":"ContainerDied","Data":"194850ad3be5b6c23cae51035510b98558ab2470d5404be2c5f8ce256f06640b"} Jul 08 04:19:57.484432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.484035 2582 scope.go:117] "RemoveContainer" containerID="0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b" Jul 08 04:19:57.493529 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.493492 2582 scope.go:117] "RemoveContainer" containerID="0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b" Jul 08 04:19:57.493897 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:19:57.493873 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b\": container with ID starting with 0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b not found: ID does not exist" containerID="0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b" Jul 08 04:19:57.493985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.493921 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b"} err="failed to get container status \"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b\": rpc error: code = NotFound desc = could not find container \"0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b\": container with ID starting with 0827520b4e8b8a25f4187c309d9594f259faadc028702edc3fa10877ee52c29b not found: ID does not exist" Jul 08 04:19:57.501943 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.501838 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:57.504869 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:57.504843 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66486b595d-spj56"] Jul 08 04:19:59.446549 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:19:59.446511 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" path="/var/lib/kubelet/pods/0a27af6c-5203-43b0-9fc2-b1c55bdc8108/volumes" Jul 08 04:20:11.632294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:11.632252 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:11.649422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:11.649393 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:12.608224 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:12.608194 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:25.330880 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.330844 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331153 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="alertmanager" containerID="cri-o://cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e" gracePeriod=120 Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331215 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-metric" containerID="cri-o://080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702" gracePeriod=120 Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331236 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-web" containerID="cri-o://c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9" gracePeriod=120 Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331260 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="prom-label-proxy" containerID="cri-o://c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c" gracePeriod=120 Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331318 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy" containerID="cri-o://d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73" gracePeriod=120 Jul 08 04:20:25.331503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.331323 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="config-reloader" containerID="cri-o://fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487" gracePeriod=120 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664823 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c" exitCode=0 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664849 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702" exitCode=0 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664855 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73" exitCode=0 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664861 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487" exitCode=0 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664866 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e" exitCode=0 Jul 08 04:20:25.664907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664892 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c"} Jul 08 04:20:25.665200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664963 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702"} Jul 08 04:20:25.665200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.664991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73"} Jul 08 04:20:25.665200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.665020 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487"} Jul 08 04:20:25.665200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:25.665047 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e"} Jul 08 04:20:26.651041 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.651017 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:20:26.672804 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.672722 2582 generic.go:356] "Generic (PLEG): container finished" podID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerID="c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9" exitCode=0 Jul 08 04:20:26.672804 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.672790 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9"} Jul 08 04:20:26.672966 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.672820 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 04:20:26.672966 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.672865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13b4296d-c54b-40b7-b3f0-d7ef6011bca8","Type":"ContainerDied","Data":"74376edcf8bb70f248cbf7c66093b05bb146b6d270a8b42ec56cda6b7d3b9ab5"} Jul 08 04:20:26.672966 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.672880 2582 scope.go:117] "RemoveContainer" containerID="c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c" Jul 08 04:20:26.682573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.682554 2582 scope.go:117] "RemoveContainer" containerID="080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702" Jul 08 04:20:26.693643 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.693620 2582 scope.go:117] "RemoveContainer" containerID="d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73" Jul 08 04:20:26.698038 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n749g\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698062 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698093 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698115 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698168 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698157 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698197 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698229 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698266 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698305 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698377 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698358 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698630 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698416 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698630 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698463 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698630 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698515 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls\") pod \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\" (UID: \"13b4296d-c54b-40b7-b3f0-d7ef6011bca8\") " Jul 08 04:20:26.698796 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.698641 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:20:26.699551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.699034 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-main-db\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.699551 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.699088 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:26.702200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.702158 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:26.704919 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.704581 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.705018 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.704913 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g" (OuterVolumeSpecName: "kube-api-access-n749g") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "kube-api-access-n749g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.705661 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.705690 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.706057 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume" (OuterVolumeSpecName: "config-volume") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.706378 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.706725 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out" (OuterVolumeSpecName: "config-out") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:20:26.712137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.711408 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.712730 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.712217 2582 scope.go:117] "RemoveContainer" containerID="c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9" Jul 08 04:20:26.712730 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.712357 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.724323 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.724284 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config" (OuterVolumeSpecName: "web-config") pod "13b4296d-c54b-40b7-b3f0-d7ef6011bca8" (UID: "13b4296d-c54b-40b7-b3f0-d7ef6011bca8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:26.731741 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.731716 2582 scope.go:117] "RemoveContainer" containerID="fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487" Jul 08 04:20:26.740215 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.740198 2582 scope.go:117] "RemoveContainer" containerID="cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e" Jul 08 04:20:26.747976 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.747956 2582 scope.go:117] "RemoveContainer" containerID="fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0" Jul 08 04:20:26.755498 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.755479 2582 scope.go:117] "RemoveContainer" containerID="c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c" Jul 08 04:20:26.755791 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.755755 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c\": container with ID starting with c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c not found: ID does not exist" containerID="c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c" Jul 08 04:20:26.755862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.755801 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c"} err="failed to get container status \"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c\": rpc error: code = NotFound desc = could not find container \"c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c\": container with ID starting with c3893f3d8334bfc58f0beb76cf2d6021d741b69c9b804c478a78030b3bcb0a7c not found: ID does not exist" Jul 08 04:20:26.755862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.755820 2582 scope.go:117] "RemoveContainer" containerID="080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702" Jul 08 04:20:26.758781 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.758729 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702\": container with ID starting with 080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702 not found: ID does not exist" containerID="080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702" Jul 08 04:20:26.758856 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.758818 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702"} err="failed to get container status \"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702\": rpc error: code = NotFound desc = could not find container \"080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702\": container with ID starting with 080180cdceb4892d9755ddf94265de6c46d75aece88f7041a3119651873fd702 not found: ID does not exist" Jul 08 04:20:26.758856 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.758846 2582 scope.go:117] "RemoveContainer" containerID="d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73" Jul 08 04:20:26.759242 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.759217 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73\": container with ID starting with d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73 not found: ID does not exist" containerID="d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73" Jul 08 04:20:26.759333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759251 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73"} err="failed to get container status \"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73\": rpc error: code = NotFound desc = could not find container \"d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73\": container with ID starting with d7c425851071535967d670678e0c05dc6bc491c26e36573e9417c5c1b4202c73 not found: ID does not exist" Jul 08 04:20:26.759333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759275 2582 scope.go:117] "RemoveContainer" containerID="c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9" Jul 08 04:20:26.759566 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.759545 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9\": container with ID starting with c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9 not found: ID does not exist" containerID="c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9" Jul 08 04:20:26.759615 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759572 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9"} err="failed to get container status \"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9\": rpc error: code = NotFound desc = could not find container \"c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9\": container with ID starting with c7952262179cc97c4431870c866df6ae3fd06b3334edcadcf47ec75959ead9c9 not found: ID does not exist" Jul 08 04:20:26.759615 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759589 2582 scope.go:117] "RemoveContainer" containerID="fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487" Jul 08 04:20:26.759859 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.759844 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487\": container with ID starting with fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487 not found: ID does not exist" containerID="fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487" Jul 08 04:20:26.759918 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759864 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487"} err="failed to get container status \"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487\": rpc error: code = NotFound desc = could not find container \"fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487\": container with ID starting with fca06060144d14f4ac9de66ebc490f7a47ba324290fbf1d12038e38e8cd8e487 not found: ID does not exist" Jul 08 04:20:26.759918 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.759877 2582 scope.go:117] "RemoveContainer" containerID="cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e" Jul 08 04:20:26.760126 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.760110 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e\": container with ID starting with cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e not found: ID does not exist" containerID="cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e" Jul 08 04:20:26.760167 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.760130 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e"} err="failed to get container status \"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e\": rpc error: code = NotFound desc = could not find container \"cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e\": container with ID starting with cf4596075eb9a0bc1865bde33022a8510eb95b8b9b4c74046b4a904a95cab72e not found: ID does not exist" Jul 08 04:20:26.760167 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.760145 2582 scope.go:117] "RemoveContainer" containerID="fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0" Jul 08 04:20:26.760389 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:26.760370 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0\": container with ID starting with fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0 not found: ID does not exist" containerID="fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0" Jul 08 04:20:26.760433 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.760394 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0"} err="failed to get container status \"fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0\": rpc error: code = NotFound desc = could not find container \"fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0\": container with ID starting with fe74faee9bdcf1300b56be857eab2d32321d26048616a188c1ea75dcfe05f3e0 not found: ID does not exist" Jul 08 04:20:26.800007 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.799968 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800007 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800000 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-main-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800007 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800013 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n749g\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-kube-api-access-n749g\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800024 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-web-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800033 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-out\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800040 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-metrics-client-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800049 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800059 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-config-volume\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800068 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-cluster-tls-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800076 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800085 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.800243 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.800092 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13b4296d-c54b-40b7-b3f0-d7ef6011bca8-tls-assets\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:26.997852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:26.997816 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:20:27.003036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:27.002952 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 04:20:27.447304 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:27.447277 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" path="/var/lib/kubelet/pods/13b4296d-c54b-40b7-b3f0-d7ef6011bca8/volumes" Jul 08 04:20:29.642419 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642382 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642792 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="prometheus" containerID="cri-o://d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1" gracePeriod=600 Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642834 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy" containerID="cri-o://d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c" gracePeriod=600 Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642862 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-web" containerID="cri-o://19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3" gracePeriod=600 Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642868 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="config-reloader" containerID="cri-o://1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341" gracePeriod=600 Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642839 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3" gracePeriod=600 Jul 08 04:20:29.642952 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:29.642843 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="thanos-sidecar" containerID="cri-o://6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85" gracePeriod=600 Jul 08 04:20:30.700360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700327 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3" exitCode=0 Jul 08 04:20:30.700360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700352 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c" exitCode=0 Jul 08 04:20:30.700360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700358 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85" exitCode=0 Jul 08 04:20:30.700360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700364 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341" exitCode=0 Jul 08 04:20:30.700360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700370 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1" exitCode=0 Jul 08 04:20:30.700896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3"} Jul 08 04:20:30.700896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700460 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c"} Jul 08 04:20:30.700896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700488 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85"} Jul 08 04:20:30.700896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700514 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341"} Jul 08 04:20:30.700896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.700539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1"} Jul 08 04:20:30.939229 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:30.939202 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.047064 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047025 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047064 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047065 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047135 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt7hr\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047210 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047262 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047292 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047321 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047376 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047408 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047437 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047467 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047500 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047534 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047568 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047605 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047647 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.047681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047677 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.048178 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047715 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.048178 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047786 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.048178 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.047839 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2\") pod \"549329d3-9863-4cf0-a98c-dcc9c25d556c\" (UID: \"549329d3-9863-4cf0-a98c-dcc9c25d556c\") " Jul 08 04:20:31.048548 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.048515 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.048872 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.048837 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:20:31.049865 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.049747 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.050149 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050117 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr" (OuterVolumeSpecName: "kube-api-access-nt7hr") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "kube-api-access-nt7hr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:20:31.050307 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050283 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.050501 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050479 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.050643 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050622 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config" (OuterVolumeSpecName: "config") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.050718 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050698 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.050807 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050741 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.050916 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.050889 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.051641 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.051611 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out" (OuterVolumeSpecName: "config-out") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:20:31.051948 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.051898 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:20:31.052258 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.052164 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.052258 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.052220 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.052532 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.052506 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.052713 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.052682 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.052822 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.052739 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:20:31.053051 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.053032 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.053252 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.053230 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.065823 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.065800 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config" (OuterVolumeSpecName: "web-config") pod "549329d3-9863-4cf0-a98c-dcc9c25d556c" (UID: "549329d3-9863-4cf0-a98c-dcc9c25d556c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:20:31.149203 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149165 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-metrics-client-certs\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149203 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149195 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-kube-rbac-proxy\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149203 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149206 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149219 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-grpc-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149228 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149239 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149247 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149255 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149263 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149272 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149280 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149288 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt7hr\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-kube-api-access-nt7hr\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149297 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149305 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/549329d3-9863-4cf0-a98c-dcc9c25d556c-tls-assets\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149312 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-prometheus-k8s-db\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149321 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149331 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/549329d3-9863-4cf0-a98c-dcc9c25d556c-config-out\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149340 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/549329d3-9863-4cf0-a98c-dcc9c25d556c-web-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149349 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-metrics-client-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.149448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.149357 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/549329d3-9863-4cf0-a98c-dcc9c25d556c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:20:31.708436 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.708346 2582 generic.go:356] "Generic (PLEG): container finished" podID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerID="19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3" exitCode=0 Jul 08 04:20:31.708923 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.708437 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.708923 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.708431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3"} Jul 08 04:20:31.708923 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.708578 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"549329d3-9863-4cf0-a98c-dcc9c25d556c","Type":"ContainerDied","Data":"d8c2ef4e736c3b6b41c51905e5ee683fb7bb00243c9a56904a2cc7f3bf4343ee"} Jul 08 04:20:31.708923 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.708612 2582 scope.go:117] "RemoveContainer" containerID="7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3" Jul 08 04:20:31.722859 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.722838 2582 scope.go:117] "RemoveContainer" containerID="d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c" Jul 08 04:20:31.730852 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.730832 2582 scope.go:117] "RemoveContainer" containerID="19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3" Jul 08 04:20:31.735528 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.735504 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:31.739893 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.739868 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:31.743772 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.743743 2582 scope.go:117] "RemoveContainer" containerID="6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85" Jul 08 04:20:31.750949 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.750927 2582 scope.go:117] "RemoveContainer" containerID="1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341" Jul 08 04:20:31.758295 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.758278 2582 scope.go:117] "RemoveContainer" containerID="d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1" Jul 08 04:20:31.767128 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.766945 2582 scope.go:117] "RemoveContainer" containerID="ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb" Jul 08 04:20:31.774995 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.774973 2582 scope.go:117] "RemoveContainer" containerID="7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3" Jul 08 04:20:31.775291 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.775270 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3\": container with ID starting with 7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3 not found: ID does not exist" containerID="7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3" Jul 08 04:20:31.775352 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.775301 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3"} err="failed to get container status \"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3\": rpc error: code = NotFound desc = could not find container \"7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3\": container with ID starting with 7e2871410ae61db2adc3adbcbfa6f6af9c9f919d4d38fc8f3ccae9c4460236c3 not found: ID does not exist" Jul 08 04:20:31.775352 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.775323 2582 scope.go:117] "RemoveContainer" containerID="d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c" Jul 08 04:20:31.775616 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.775597 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c\": container with ID starting with d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c not found: ID does not exist" containerID="d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c" Jul 08 04:20:31.775664 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.775622 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c"} err="failed to get container status \"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c\": rpc error: code = NotFound desc = could not find container \"d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c\": container with ID starting with d29268132002efadd23f7c2e2159283af4de573a2f2a7f7b09035bbf0b8b338c not found: ID does not exist" Jul 08 04:20:31.775664 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.775642 2582 scope.go:117] "RemoveContainer" containerID="19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3" Jul 08 04:20:31.775974 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.775954 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3\": container with ID starting with 19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3 not found: ID does not exist" containerID="19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3" Jul 08 04:20:31.776030 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.775981 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3"} err="failed to get container status \"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3\": rpc error: code = NotFound desc = could not find container \"19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3\": container with ID starting with 19eee9ca7031594c8c8528955883a337e505f18535867a960ad59f4aa653afc3 not found: ID does not exist" Jul 08 04:20:31.776030 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776005 2582 scope.go:117] "RemoveContainer" containerID="6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85" Jul 08 04:20:31.776244 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.776228 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85\": container with ID starting with 6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85 not found: ID does not exist" containerID="6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85" Jul 08 04:20:31.776289 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776247 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85"} err="failed to get container status \"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85\": rpc error: code = NotFound desc = could not find container \"6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85\": container with ID starting with 6aaa3562a5395a9651fc787cf30e301cdb9aa44a5766fa7188f6f50b192d7e85 not found: ID does not exist" Jul 08 04:20:31.776289 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776259 2582 scope.go:117] "RemoveContainer" containerID="1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341" Jul 08 04:20:31.776454 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.776438 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341\": container with ID starting with 1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341 not found: ID does not exist" containerID="1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341" Jul 08 04:20:31.776516 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776459 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341"} err="failed to get container status \"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341\": rpc error: code = NotFound desc = could not find container \"1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341\": container with ID starting with 1f771d29a687ae02e459b8a04fd2529ff1a69274cbca210df8259593b0317341 not found: ID does not exist" Jul 08 04:20:31.776516 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776501 2582 scope.go:117] "RemoveContainer" containerID="d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1" Jul 08 04:20:31.776790 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.776755 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1\": container with ID starting with d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1 not found: ID does not exist" containerID="d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1" Jul 08 04:20:31.776873 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776795 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1"} err="failed to get container status \"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1\": rpc error: code = NotFound desc = could not find container \"d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1\": container with ID starting with d6dea1e314e0dde9862d1bc2c7766ded690d31b51a52f066159fab6cff6377f1 not found: ID does not exist" Jul 08 04:20:31.776873 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.776816 2582 scope.go:117] "RemoveContainer" containerID="ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb" Jul 08 04:20:31.777060 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:20:31.777044 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb\": container with ID starting with ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb not found: ID does not exist" containerID="ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb" Jul 08 04:20:31.777108 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:31.777064 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb"} err="failed to get container status \"ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb\": rpc error: code = NotFound desc = could not find container \"ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb\": container with ID starting with ea729326768c961bacdc5d08241382e3c1e87e4218d35448f7aa84df307fb8bb not found: ID does not exist" Jul 08 04:20:33.446036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:33.445992 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" path="/var/lib/kubelet/pods/549329d3-9863-4cf0-a98c-dcc9c25d556c/volumes" Jul 08 04:20:45.339083 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:20:45.339039 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:21:10.363171 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.363123 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-84bdc9c4cd-9rlmw" podUID="6523611c-7bf5-41f0-a21b-6127aac61cc7" containerName="console" containerID="cri-o://8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c" gracePeriod=15 Jul 08 04:21:10.658996 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.658970 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:21:10.706587 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706553 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.706788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706597 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.706788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706617 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.706788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706646 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mjf5\" (UniqueName: \"kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.706788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706713 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.706788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706776 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.707071 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.706821 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert\") pod \"6523611c-7bf5-41f0-a21b-6127aac61cc7\" (UID: \"6523611c-7bf5-41f0-a21b-6127aac61cc7\") " Jul 08 04:21:10.707132 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.707071 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:21:10.707357 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.707325 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config" (OuterVolumeSpecName: "console-config") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:21:10.707511 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.707397 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:21:10.707511 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.707406 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:21:10.709089 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.709064 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5" (OuterVolumeSpecName: "kube-api-access-2mjf5") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "kube-api-access-2mjf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:21:10.709342 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.709323 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:21:10.709415 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.709362 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6523611c-7bf5-41f0-a21b-6127aac61cc7" (UID: "6523611c-7bf5-41f0-a21b-6127aac61cc7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:21:10.808273 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808228 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808273 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808268 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-oauth-serving-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808273 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808278 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-oauth-config\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808273 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808287 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-service-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808296 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6523611c-7bf5-41f0-a21b-6127aac61cc7-console-serving-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808304 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mjf5\" (UniqueName: \"kubernetes.io/projected/6523611c-7bf5-41f0-a21b-6127aac61cc7-kube-api-access-2mjf5\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.808552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.808315 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6523611c-7bf5-41f0-a21b-6127aac61cc7-trusted-ca-bundle\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:21:10.908842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.908727 2582 generic.go:356] "Generic (PLEG): container finished" podID="6523611c-7bf5-41f0-a21b-6127aac61cc7" containerID="8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c" exitCode=0 Jul 08 04:21:10.908842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.908813 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bdc9c4cd-9rlmw" Jul 08 04:21:10.908842 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.908808 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bdc9c4cd-9rlmw" event={"ID":"6523611c-7bf5-41f0-a21b-6127aac61cc7","Type":"ContainerDied","Data":"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c"} Jul 08 04:21:10.909066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.908873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bdc9c4cd-9rlmw" event={"ID":"6523611c-7bf5-41f0-a21b-6127aac61cc7","Type":"ContainerDied","Data":"1656ee92ec387eb9a28108e43649fdc17bed76940970dccfb91a489f4316cd78"} Jul 08 04:21:10.909066 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.908897 2582 scope.go:117] "RemoveContainer" containerID="8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c" Jul 08 04:21:10.917864 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.917845 2582 scope.go:117] "RemoveContainer" containerID="8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c" Jul 08 04:21:10.918148 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:21:10.918116 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c\": container with ID starting with 8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c not found: ID does not exist" containerID="8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c" Jul 08 04:21:10.918230 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.918154 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c"} err="failed to get container status \"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c\": rpc error: code = NotFound desc = could not find container \"8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c\": container with ID starting with 8573ec0167bccaedd144bea583c07f4bee84b79c80c09393027e14678c59ce0c not found: ID does not exist" Jul 08 04:21:10.929700 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.929676 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:21:10.933345 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:10.933325 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84bdc9c4cd-9rlmw"] Jul 08 04:21:11.445858 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:21:11.445826 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6523611c-7bf5-41f0-a21b-6127aac61cc7" path="/var/lib/kubelet/pods/6523611c-7bf5-41f0-a21b-6127aac61cc7/volumes" Jul 08 04:22:35.351819 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:22:35.351787 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:22:35.358598 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:22:35.358575 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:22:35.360293 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:22:35.360254 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:22:35.364803 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:22:35.364781 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 04:22:35.366973 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:22:35.366956 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:24:01.601964 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.601883 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8"] Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602241 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602252 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602263 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602268 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602275 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-metric" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602280 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-metric" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602288 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="prometheus" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602293 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="prometheus" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602303 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="thanos-sidecar" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602308 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="thanos-sidecar" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602316 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6523611c-7bf5-41f0-a21b-6127aac61cc7" containerName="console" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602320 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6523611c-7bf5-41f0-a21b-6127aac61cc7" containerName="console" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602328 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="init-config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602333 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="init-config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602339 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602344 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602350 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602356 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602362 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" containerName="console" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602367 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" containerName="console" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602375 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602380 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602387 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602394 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602399 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-thanos" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602404 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-thanos" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602412 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="init-config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602417 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="init-config-reloader" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602423 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="prom-label-proxy" Jul 08 04:24:01.602413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602429 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="prom-label-proxy" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602438 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="alertmanager" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602443 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="alertmanager" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602449 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602453 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602503 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-thanos" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602511 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="de847b82-b567-41b9-9f9e-92ee2815ca85" containerName="registry" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602516 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a27af6c-5203-43b0-9fc2-b1c55bdc8108" containerName="console" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602523 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="thanos-sidecar" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602529 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602535 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="kube-rbac-proxy" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602540 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6523611c-7bf5-41f0-a21b-6127aac61cc7" containerName="console" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602545 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="config-reloader" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602550 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-web" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602555 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602561 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="alertmanager" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602567 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="config-reloader" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602572 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="kube-rbac-proxy-metric" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602578 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b4296d-c54b-40b7-b3f0-d7ef6011bca8" containerName="prom-label-proxy" Jul 08 04:24:01.603427 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.602583 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="549329d3-9863-4cf0-a98c-dcc9c25d556c" containerName="prometheus" Jul 08 04:24:01.605990 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.605972 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.608642 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.608620 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 04:24:01.608642 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.608628 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qrljm\"" Jul 08 04:24:01.608990 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.608972 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 04:24:01.609714 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.609698 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 04:24:01.609791 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.609721 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 04:24:01.626999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.626964 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8"] Jul 08 04:24:01.660126 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.660087 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.660334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.660136 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.660334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.660162 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.660334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.660249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.660334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.660321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w57zs\" (UniqueName: \"kubernetes.io/projected/fbc8d43e-26dc-4f82-8336-71df82123acb-kube-api-access-w57zs\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.761381 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.761342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.761584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.761394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w57zs\" (UniqueName: \"kubernetes.io/projected/fbc8d43e-26dc-4f82-8336-71df82123acb-kube-api-access-w57zs\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.761584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.761516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.761584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.761552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.761584 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.761575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.762046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.762016 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.763883 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.763860 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc8d43e-26dc-4f82-8336-71df82123acb-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.764039 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.764021 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.764104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.764075 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc8d43e-26dc-4f82-8336-71df82123acb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.778669 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.778649 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w57zs\" (UniqueName: \"kubernetes.io/projected/fbc8d43e-26dc-4f82-8336-71df82123acb-kube-api-access-w57zs\") pod \"opendatahub-operator-controller-manager-7f76f945bd-w65q8\" (UID: \"fbc8d43e-26dc-4f82-8336-71df82123acb\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:01.916163 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:01.916083 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:02.055715 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:02.055590 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8"] Jul 08 04:24:02.058680 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:24:02.058653 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc8d43e_26dc_4f82_8336_71df82123acb.slice/crio-bea63cb3ddc5d26e0f2a1a12187479150f755493b5ce5b700f0bda2bc16df80a WatchSource:0}: Error finding container bea63cb3ddc5d26e0f2a1a12187479150f755493b5ce5b700f0bda2bc16df80a: Status 404 returned error can't find the container with id bea63cb3ddc5d26e0f2a1a12187479150f755493b5ce5b700f0bda2bc16df80a Jul 08 04:24:02.060432 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:02.060415 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:24:02.732223 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:02.732156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" event={"ID":"fbc8d43e-26dc-4f82-8336-71df82123acb","Type":"ContainerStarted","Data":"bea63cb3ddc5d26e0f2a1a12187479150f755493b5ce5b700f0bda2bc16df80a"} Jul 08 04:24:05.752113 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:05.752075 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" event={"ID":"fbc8d43e-26dc-4f82-8336-71df82123acb","Type":"ContainerStarted","Data":"a7cddd9257551e0e09e40af98df954b7688ba1eee6b120275234651aedaba45a"} Jul 08 04:24:06.758325 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:06.758289 2582 generic.go:356] "Generic (PLEG): container finished" podID="fbc8d43e-26dc-4f82-8336-71df82123acb" containerID="a7cddd9257551e0e09e40af98df954b7688ba1eee6b120275234651aedaba45a" exitCode=0 Jul 08 04:24:06.758788 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:06.758371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" event={"ID":"fbc8d43e-26dc-4f82-8336-71df82123acb","Type":"ContainerDied","Data":"a7cddd9257551e0e09e40af98df954b7688ba1eee6b120275234651aedaba45a"} Jul 08 04:24:07.765274 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:07.765239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" event={"ID":"fbc8d43e-26dc-4f82-8336-71df82123acb","Type":"ContainerStarted","Data":"6109e114dac23441fa537b20bcb370b16669242be24cb3dcbc397e43c1e35ee2"} Jul 08 04:24:07.765739 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:07.765342 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:07.786916 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:07.786856 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" podStartSLOduration=2.354509016 podStartE2EDuration="6.786834638s" totalImagesPullingTime="3.252761442s" totalInitContainerRuntime="1.17956418s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:07.785712891 +0000 UTC m=+392.887551837" watchObservedRunningTime="2026-07-08 04:24:07.786834638 +0000 UTC m=+392.888673587" Jul 08 04:24:18.772416 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:18.772387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-w65q8" Jul 08 04:24:27.058804 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.058746 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-sx7pg"] Jul 08 04:24:27.072933 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.072870 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-sx7pg"] Jul 08 04:24:27.073111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.072980 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.075512 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.075488 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 04:24:27.075512 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.075497 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 04:24:27.075727 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.075496 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rbnwh\"" Jul 08 04:24:27.110657 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.110620 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3380d5-c31b-47f1-9b37-96555916b75a-tmp\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.110885 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.110685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a3380d5-c31b-47f1-9b37-96555916b75a-tls-certs\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.110885 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.110797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h76ll\" (UniqueName: \"kubernetes.io/projected/8a3380d5-c31b-47f1-9b37-96555916b75a-kube-api-access-h76ll\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.211910 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.211862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h76ll\" (UniqueName: \"kubernetes.io/projected/8a3380d5-c31b-47f1-9b37-96555916b75a-kube-api-access-h76ll\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.212114 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.211931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3380d5-c31b-47f1-9b37-96555916b75a-tmp\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.212114 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.212038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a3380d5-c31b-47f1-9b37-96555916b75a-tls-certs\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.214355 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.214322 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a3380d5-c31b-47f1-9b37-96555916b75a-tmp\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.214500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.214399 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a3380d5-c31b-47f1-9b37-96555916b75a-tls-certs\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.219717 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.219691 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h76ll\" (UniqueName: \"kubernetes.io/projected/8a3380d5-c31b-47f1-9b37-96555916b75a-kube-api-access-h76ll\") pod \"kube-auth-proxy-668854d458-sx7pg\" (UID: \"8a3380d5-c31b-47f1-9b37-96555916b75a\") " pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.384125 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.384033 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" Jul 08 04:24:27.523975 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.523942 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-sx7pg"] Jul 08 04:24:27.526815 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:24:27.526781 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3380d5_c31b_47f1_9b37_96555916b75a.slice/crio-1891ad662b22e02283de090dac966943f541c945cf909a0558ec19affe80e686 WatchSource:0}: Error finding container 1891ad662b22e02283de090dac966943f541c945cf909a0558ec19affe80e686: Status 404 returned error can't find the container with id 1891ad662b22e02283de090dac966943f541c945cf909a0558ec19affe80e686 Jul 08 04:24:27.864603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:27.864531 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" event={"ID":"8a3380d5-c31b-47f1-9b37-96555916b75a","Type":"ContainerStarted","Data":"1891ad662b22e02283de090dac966943f541c945cf909a0558ec19affe80e686"} Jul 08 04:24:29.706366 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.706328 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl"] Jul 08 04:24:29.710373 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.710347 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.712977 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.712949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 04:24:29.713129 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.713052 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 04:24:29.713289 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.713263 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xsdpj\"" Jul 08 04:24:29.713393 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.713306 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 04:24:29.713680 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.713659 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 04:24:29.725362 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.725335 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl"] Jul 08 04:24:29.735751 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.735712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.735929 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.735809 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d3b1bb1-9257-4343-8abe-458b71abd410-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.735996 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.735926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.735996 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.735968 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.736091 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.736051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.736091 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.736083 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.736209 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.736186 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7jlj\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-kube-api-access-h7jlj\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837407 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7jlj\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-kube-api-access-h7jlj\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837407 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837672 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d3b1bb1-9257-4343-8abe-458b71abd410-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837672 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837672 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837672 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.837912 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.837677 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.838369 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.838347 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.840775 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.840742 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.840950 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.840906 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d3b1bb1-9257-4343-8abe-458b71abd410-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.841074 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.840947 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.841074 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.840947 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d3b1bb1-9257-4343-8abe-458b71abd410-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.850025 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.849993 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7jlj\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-kube-api-access-h7jlj\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:29.850314 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:29.850291 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d3b1bb1-9257-4343-8abe-458b71abd410-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x28pl\" (UID: \"3d3b1bb1-9257-4343-8abe-458b71abd410\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:30.022937 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:30.022892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:30.506817 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:30.506791 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl"] Jul 08 04:24:30.509487 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:24:30.509464 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3b1bb1_9257_4343_8abe_458b71abd410.slice/crio-a8dd058853ae4e9258e52957a08adc8b8871d089d0713fd7374cb512d488410d WatchSource:0}: Error finding container a8dd058853ae4e9258e52957a08adc8b8871d089d0713fd7374cb512d488410d: Status 404 returned error can't find the container with id a8dd058853ae4e9258e52957a08adc8b8871d089d0713fd7374cb512d488410d Jul 08 04:24:30.890431 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:30.890387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" event={"ID":"3d3b1bb1-9257-4343-8abe-458b71abd410","Type":"ContainerStarted","Data":"a8dd058853ae4e9258e52957a08adc8b8871d089d0713fd7374cb512d488410d"} Jul 08 04:24:30.892043 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:30.892013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" event={"ID":"8a3380d5-c31b-47f1-9b37-96555916b75a","Type":"ContainerStarted","Data":"ad855f8dfd7cca02e8fc9c8121a2a86b4a5132f2aa98ade6292955754d9cddcf"} Jul 08 04:24:30.913603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:30.913552 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-668854d458-sx7pg" podStartSLOduration=1.0289355040000001 podStartE2EDuration="3.913534675s" totalImagesPullingTime="2.884599171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:30.911861135 +0000 UTC m=+416.013700098" watchObservedRunningTime="2026-07-08 04:24:30.913534675 +0000 UTC m=+416.015373605" Jul 08 04:24:32.928960 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.927662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr"] Jul 08 04:24:32.935754 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.935729 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.940143 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.940113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rwrh2\"" Jul 08 04:24:32.946965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.946681 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr"] Jul 08 04:24:32.972808 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2d63baa-4daf-400c-82af-38759e29fe53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.972808 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972808 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2d63baa-4daf-400c-82af-38759e29fe53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972899 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972939 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcx7p\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-kube-api-access-jcx7p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.972989 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.973021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973085 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.973068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:32.973351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:32.973110 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074395 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2d63baa-4daf-400c-82af-38759e29fe53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074567 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2d63baa-4daf-400c-82af-38759e29fe53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074610 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcx7p\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-kube-api-access-jcx7p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074695 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.074935 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.074923 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.075908 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.075877 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.075908 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.075892 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2d63baa-4daf-400c-82af-38759e29fe53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.076095 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.075899 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.076184 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.076148 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.078005 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.077977 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2d63baa-4daf-400c-82af-38759e29fe53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.079021 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.078988 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2d63baa-4daf-400c-82af-38759e29fe53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.087347 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.087310 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcx7p\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-kube-api-access-jcx7p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.092156 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.092130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2d63baa-4daf-400c-82af-38759e29fe53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46n8hr\" (UID: \"e2d63baa-4daf-400c-82af-38759e29fe53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.252548 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.252520 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:33.304259 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.304210 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:24:33.304484 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.304299 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:24:33.514537 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.514455 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr"] Jul 08 04:24:33.518145 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:24:33.518110 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d63baa_4daf_400c_82af_38759e29fe53.slice/crio-1d85e88a83145e6bdc83b71d4004af75a56d4efe7eeadf4d062770f1f93219f7 WatchSource:0}: Error finding container 1d85e88a83145e6bdc83b71d4004af75a56d4efe7eeadf4d062770f1f93219f7: Status 404 returned error can't find the container with id 1d85e88a83145e6bdc83b71d4004af75a56d4efe7eeadf4d062770f1f93219f7 Jul 08 04:24:33.913030 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.912939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" event={"ID":"3d3b1bb1-9257-4343-8abe-458b71abd410","Type":"ContainerStarted","Data":"3a472b358c2286a0515817c123a781637e8e29aac731a857e17f0663ace12485"} Jul 08 04:24:33.913207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.913045 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:33.914177 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.914153 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" event={"ID":"e2d63baa-4daf-400c-82af-38759e29fe53","Type":"ContainerStarted","Data":"1d85e88a83145e6bdc83b71d4004af75a56d4efe7eeadf4d062770f1f93219f7"} Jul 08 04:24:33.982280 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:33.982215 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" podStartSLOduration=2.189881312 podStartE2EDuration="4.982197014s" totalImagesPullingTime="2.792315702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:33.976358865 +0000 UTC m=+419.078197819" watchObservedRunningTime="2026-07-08 04:24:33.982197014 +0000 UTC m=+419.084035942" Jul 08 04:24:34.921568 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:34.921543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x28pl" Jul 08 04:24:36.943475 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:36.943432 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:24:36.943748 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:36.943515 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:24:36.943748 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:36.943567 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:24:37.936473 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:37.936431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" event={"ID":"e2d63baa-4daf-400c-82af-38759e29fe53","Type":"ContainerStarted","Data":"b51c853858bc39110338a5401bc6fcd90e4e23f024c618a5893939251a59e319"} Jul 08 04:24:37.981899 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:37.981840 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" podStartSLOduration=2.559267792 podStartE2EDuration="5.98182231s" totalImagesPullingTime="3.422554518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:37.978813067 +0000 UTC m=+423.080652012" watchObservedRunningTime="2026-07-08 04:24:37.98182231 +0000 UTC m=+423.083661238" Jul 08 04:24:38.253553 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:38.253521 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:38.258247 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:38.258221 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:38.941333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:38.941301 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:24:38.942462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:24:38.942442 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46n8hr" Jul 08 04:25:35.401525 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:25:35.401491 2582 scope.go:117] "RemoveContainer" containerID="33b20395acea1b0c9b42768b3aad35806a4f67ed32cc966ced8f64bb12603730" Jul 08 04:26:25.681011 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.680973 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9"] Jul 08 04:26:25.684862 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.684837 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.687519 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.687489 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 04:26:25.687519 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.687510 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p6hqn\"" Jul 08 04:26:25.687703 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.687552 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 04:26:25.699554 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.699533 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9"] Jul 08 04:26:25.769689 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.769650 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqmdn\" (UniqueName: \"kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.769934 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.769712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.871187 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.871143 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqmdn\" (UniqueName: \"kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.871371 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.871206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.871604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.871582 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.880620 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.880589 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqmdn\" (UniqueName: \"kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:25.995853 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:25.995824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:26.149429 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:26.149404 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9"] Jul 08 04:26:26.151528 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:26:26.151492 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21b3e53_b2a7_43a2_a9b2_9be0de0f2573.slice/crio-200b84a536287949fa272a788430ad3d543d808cb8893cc2108a43f0b8409e28 WatchSource:0}: Error finding container 200b84a536287949fa272a788430ad3d543d808cb8893cc2108a43f0b8409e28: Status 404 returned error can't find the container with id 200b84a536287949fa272a788430ad3d543d808cb8893cc2108a43f0b8409e28 Jul 08 04:26:26.395368 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:26.395274 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9"] Jul 08 04:26:26.401515 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:26.401482 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9"] Jul 08 04:26:31.516710 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.516664 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" containerName="manager" containerID="cri-o://6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7" gracePeriod=2 Jul 08 04:26:31.770723 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.770659 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:31.773014 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:26:31.772989 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" is forbidden: User \"system:node:ip-10-0-142-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-181.ec2.internal' and this object" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:31.832648 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.832617 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume\") pod \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " Jul 08 04:26:31.832831 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.832698 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqmdn\" (UniqueName: \"kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn\") pod \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\" (UID: \"d21b3e53-b2a7-43a2-a9b2-9be0de0f2573\") " Jul 08 04:26:31.832972 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.832944 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" (UID: "d21b3e53-b2a7-43a2-a9b2-9be0de0f2573"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:26:31.834941 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.834912 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn" (OuterVolumeSpecName: "kube-api-access-lqmdn") pod "d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" (UID: "d21b3e53-b2a7-43a2-a9b2-9be0de0f2573"). InnerVolumeSpecName "kube-api-access-lqmdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:26:31.934442 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.934405 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-extensions-socket-volume\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:26:31.934442 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:31.934434 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqmdn\" (UniqueName: \"kubernetes.io/projected/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573-kube-api-access-lqmdn\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:26:32.522537 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:32.522501 2582 generic.go:356] "Generic (PLEG): container finished" podID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" containerID="6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7" exitCode=2 Jul 08 04:26:32.522537 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:32.522543 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:32.523131 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:32.522599 2582 scope.go:117] "RemoveContainer" containerID="6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7" Jul 08 04:26:32.525323 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:26:32.525285 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" is forbidden: User \"system:node:ip-10-0-142-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-181.ec2.internal' and this object" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:32.531707 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:32.531691 2582 scope.go:117] "RemoveContainer" containerID="6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7" Jul 08 04:26:32.531965 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:26:32.531947 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7\": container with ID starting with 6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7 not found: ID does not exist" containerID="6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7" Jul 08 04:26:32.532044 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:32.531972 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7"} err="failed to get container status \"6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7\": rpc error: code = NotFound desc = could not find container \"6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7\": container with ID starting with 6f4d14060457ab7a7fbf28ce4a7b15f57b89b680dc74bea72ba76751d59a8dc7 not found: ID does not exist" Jul 08 04:26:32.534300 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:26:32.534281 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m7br9\" is forbidden: User \"system:node:ip-10-0-142-181.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-181.ec2.internal' and this object" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m7br9" Jul 08 04:26:33.446456 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:33.446419 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" path="/var/lib/kubelet/pods/d21b3e53-b2a7-43a2-a9b2-9be0de0f2573/volumes" Jul 08 04:26:58.247872 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.247830 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:26:58.248364 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.248246 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" containerName="manager" Jul 08 04:26:58.248364 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.248258 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" containerName="manager" Jul 08 04:26:58.248364 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.248334 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21b3e53-b2a7-43a2-a9b2-9be0de0f2573" containerName="manager" Jul 08 04:26:58.253075 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.253056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.255670 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.255645 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 04:26:58.255670 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.255664 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 04:26:58.255879 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.255724 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p6hqn\"" Jul 08 04:26:58.266504 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.266484 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:26:58.290673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.290636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.290849 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.290688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rkk4\" (UniqueName: \"kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.391882 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.391848 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.392067 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.391907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rkk4\" (UniqueName: \"kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.392281 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.392261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.400392 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.400363 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rkk4\" (UniqueName: \"kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4\") pod \"kuadrant-operator-controller-manager-86878c769f-kmbf2\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.564294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.564194 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:58.708822 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:58.708798 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:26:58.711099 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:26:58.711071 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f54ae0_8d53_43fe_bb06_f48928e9f1a9.slice/crio-5b21fb1c647987731c8ccf51fe78cd5479e31162ce2111984b5132addf92334c WatchSource:0}: Error finding container 5b21fb1c647987731c8ccf51fe78cd5479e31162ce2111984b5132addf92334c: Status 404 returned error can't find the container with id 5b21fb1c647987731c8ccf51fe78cd5479e31162ce2111984b5132addf92334c Jul 08 04:26:59.672094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:59.672057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" event={"ID":"00f54ae0-8d53-43fe-bb06-f48928e9f1a9","Type":"ContainerStarted","Data":"6565d58e060f2cf4c6e05f592522f03e7ac2bb8202f3f276782ccc33e0ec7375"} Jul 08 04:26:59.672617 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:59.672115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" event={"ID":"00f54ae0-8d53-43fe-bb06-f48928e9f1a9","Type":"ContainerStarted","Data":"5b21fb1c647987731c8ccf51fe78cd5479e31162ce2111984b5132addf92334c"} Jul 08 04:26:59.672617 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:59.672242 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:26:59.693653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:26:59.693595 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" podStartSLOduration=1.693579647 podStartE2EDuration="1.693579647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:26:59.692295598 +0000 UTC m=+564.794134554" watchObservedRunningTime="2026-07-08 04:26:59.693579647 +0000 UTC m=+564.795418574" Jul 08 04:27:10.680111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:27:10.680077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:27:35.390336 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:27:35.390291 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:27:35.393840 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:27:35.393820 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:27:35.398500 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:27:35.398476 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:27:35.401875 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:27:35.401857 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:28:34.484973 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.484940 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mrwb4"] Jul 08 04:28:34.488382 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.488365 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.491118 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.491092 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rbn45\"" Jul 08 04:28:34.491238 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.491152 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 04:28:34.499589 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.499562 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mrwb4"] Jul 08 04:28:34.517102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.517071 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2kdb5"] Jul 08 04:28:34.520937 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.520914 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.532678 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.532642 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2kdb5"] Jul 08 04:28:34.576543 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.576497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.576744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.576633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8dft\" (UniqueName: \"kubernetes.io/projected/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-kube-api-access-q8dft\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.576744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.576666 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-plugins-config-volume\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.576744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.576715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsdzb\" (UniqueName: \"kubernetes.io/projected/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-kube-api-access-wsdzb\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.677260 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.677225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8dft\" (UniqueName: \"kubernetes.io/projected/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-kube-api-access-q8dft\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.677260 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.677261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-plugins-config-volume\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.677528 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.677300 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsdzb\" (UniqueName: \"kubernetes.io/projected/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-kube-api-access-wsdzb\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.677528 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.677342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.678081 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.678059 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.678136 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.678060 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-plugins-config-volume\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.691073 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.691045 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8dft\" (UniqueName: \"kubernetes.io/projected/31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3-kube-api-access-q8dft\") pod \"payload-pre-processing-7964457696-mrwb4\" (UID: \"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3\") " pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.691224 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.691167 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsdzb\" (UniqueName: \"kubernetes.io/projected/9398e4ee-f27b-4f53-9e80-7c938d7d5cb2-kube-api-access-wsdzb\") pod \"payload-processing-58f86c7798-2kdb5\" (UID: \"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2\") " pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.800448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.800408 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:34.834060 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.834025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:34.956546 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.956503 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mrwb4"] Jul 08 04:28:34.958713 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:28:34.958675 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e8e8d5_80a7_41ff_a99d_0d1aef8a79f3.slice/crio-d0ab0e0fd790a127b1d33b84e197c18b2723f84d25c60010f065cd88bb47dea2 WatchSource:0}: Error finding container d0ab0e0fd790a127b1d33b84e197c18b2723f84d25c60010f065cd88bb47dea2: Status 404 returned error can't find the container with id d0ab0e0fd790a127b1d33b84e197c18b2723f84d25c60010f065cd88bb47dea2 Jul 08 04:28:34.996167 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:34.996141 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2kdb5"] Jul 08 04:28:34.998289 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:28:34.998262 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9398e4ee_f27b_4f53_9e80_7c938d7d5cb2.slice/crio-19f918883b994c1c736f69d29c504ad9e90ed176a3bf98aa28aac5952516cd48 WatchSource:0}: Error finding container 19f918883b994c1c736f69d29c504ad9e90ed176a3bf98aa28aac5952516cd48: Status 404 returned error can't find the container with id 19f918883b994c1c736f69d29c504ad9e90ed176a3bf98aa28aac5952516cd48 Jul 08 04:28:35.168858 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:35.168742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" event={"ID":"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2","Type":"ContainerStarted","Data":"19f918883b994c1c736f69d29c504ad9e90ed176a3bf98aa28aac5952516cd48"} Jul 08 04:28:35.170107 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:35.170081 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" event={"ID":"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3","Type":"ContainerStarted","Data":"d0ab0e0fd790a127b1d33b84e197c18b2723f84d25c60010f065cd88bb47dea2"} Jul 08 04:28:37.182721 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.182682 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" event={"ID":"9398e4ee-f27b-4f53-9e80-7c938d7d5cb2","Type":"ContainerStarted","Data":"ea929a6e3d204835ae379d8603b31f8227bcb89e9a4ff5e1bb2e71b08ee87ba2"} Jul 08 04:28:37.183188 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.182821 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:37.184459 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.184434 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" event={"ID":"31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3","Type":"ContainerStarted","Data":"13b4c2722cd09dda924cd27e7ecd7711069a70480c4e4fa127f5a827a742d062"} Jul 08 04:28:37.184601 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.184484 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:37.199231 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.199182 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" podStartSLOduration=1.8549310829999999 podStartE2EDuration="3.199168046s" totalImagesPullingTime="1.344236963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:28:37.197174359 +0000 UTC m=+662.299013312" watchObservedRunningTime="2026-07-08 04:28:37.199168046 +0000 UTC m=+662.301006974" Jul 08 04:28:37.213592 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:37.213546 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" podStartSLOduration=1.833434849 podStartE2EDuration="3.213530344s" totalImagesPullingTime="1.380095495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:28:37.211725503 +0000 UTC m=+662.313564433" watchObservedRunningTime="2026-07-08 04:28:37.213530344 +0000 UTC m=+662.315369271" Jul 08 04:28:48.193547 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:48.193509 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-2kdb5" Jul 08 04:28:48.193943 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:48.193667 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-mrwb4" Jul 08 04:28:50.291831 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.291796 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd"] Jul 08 04:28:50.297151 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.297130 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.310945 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.310919 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 04:28:50.311076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.311061 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 04:28:50.311226 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.311206 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 04:28:50.311396 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.311361 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zrrld\"" Jul 08 04:28:50.312199 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.312175 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd"] Jul 08 04:28:50.435751 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzr6m\" (UniqueName: \"kubernetes.io/projected/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-kube-api-access-mzr6m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.435965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435805 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.435965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.435965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435869 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.435965 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.436145 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.435991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.537605 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.537566 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzr6m\" (UniqueName: \"kubernetes.io/projected/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-kube-api-access-mzr6m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.537819 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.537798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.537895 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.537850 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.537956 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.537943 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.538020 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.538004 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.538079 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.538048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.538311 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.538289 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.538429 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.538357 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.538429 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.538412 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.540268 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.540236 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.540515 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.540497 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.546663 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.546602 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzr6m\" (UniqueName: \"kubernetes.io/projected/58a66c4c-a1e7-421f-bbe8-da61645e1a5e-kube-api-access-mzr6m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd\" (UID: \"58a66c4c-a1e7-421f-bbe8-da61645e1a5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.608372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.608337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:28:50.752213 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:50.752183 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd"] Jul 08 04:28:50.754029 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:28:50.754002 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a66c4c_a1e7_421f_bbe8_da61645e1a5e.slice/crio-639305b610733b8ebb6fea2daab3b5e6c227d2e1926af54fa1976c5c447da5b7 WatchSource:0}: Error finding container 639305b610733b8ebb6fea2daab3b5e6c227d2e1926af54fa1976c5c447da5b7: Status 404 returned error can't find the container with id 639305b610733b8ebb6fea2daab3b5e6c227d2e1926af54fa1976c5c447da5b7 Jul 08 04:28:51.261917 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:51.261873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" event={"ID":"58a66c4c-a1e7-421f-bbe8-da61645e1a5e","Type":"ContainerStarted","Data":"639305b610733b8ebb6fea2daab3b5e6c227d2e1926af54fa1976c5c447da5b7"} Jul 08 04:28:52.269094 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:52.269055 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" event={"ID":"58a66c4c-a1e7-421f-bbe8-da61645e1a5e","Type":"ContainerStarted","Data":"6fc53fb6d212fdf771f9adddfb31484b2aabbaf82cd1ea94ba18cedc68db7abb"} Jul 08 04:28:52.290360 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:28:52.290295 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" podStartSLOduration=1.200596154 podStartE2EDuration="2.290277454s" totalImagesPullingTime="1.0896813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:28:52.28818247 +0000 UTC m=+677.390021411" watchObservedRunningTime="2026-07-08 04:28:52.290277454 +0000 UTC m=+677.392116382" Jul 08 04:29:00.609388 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:29:00.609352 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:29:00.609795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:29:00.609437 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:29:00.621430 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:29:00.621404 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:29:01.327622 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:29:01.327587 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd" Jul 08 04:30:00.147051 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.146947 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:30:00.150774 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.150745 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:30:00.153207 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.153183 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 04:30:00.154131 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.154113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 04:30:00.154253 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.154141 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 04:30:00.162046 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.162018 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:30:00.248413 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.248374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9nqz\" (UniqueName: \"kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz\") pod \"maas-api-key-cleanup-29724750-88w85\" (UID: \"fee225c7-4afd-4dc3-9a9f-a118b0e947e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:30:00.349956 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.349918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9nqz\" (UniqueName: \"kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz\") pod \"maas-api-key-cleanup-29724750-88w85\" (UID: \"fee225c7-4afd-4dc3-9a9f-a118b0e947e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:30:00.358847 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.358828 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9nqz\" (UniqueName: \"kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz\") pod \"maas-api-key-cleanup-29724750-88w85\" (UID: \"fee225c7-4afd-4dc3-9a9f-a118b0e947e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:30:00.463249 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.463145 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:30:00.599196 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.599173 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:30:00.601940 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:30:00.601914 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee225c7_4afd_4dc3_9a9f_a118b0e947e5.slice/crio-c55d8d2dcb95a2ac5717e1341087de77cd39cfd2e7e7bfeb778b5305becf24f0 WatchSource:0}: Error finding container c55d8d2dcb95a2ac5717e1341087de77cd39cfd2e7e7bfeb778b5305becf24f0: Status 404 returned error can't find the container with id c55d8d2dcb95a2ac5717e1341087de77cd39cfd2e7e7bfeb778b5305becf24f0 Jul 08 04:30:00.603944 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.603928 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:30:00.621419 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:00.621394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" event={"ID":"fee225c7-4afd-4dc3-9a9f-a118b0e947e5","Type":"ContainerStarted","Data":"c55d8d2dcb95a2ac5717e1341087de77cd39cfd2e7e7bfeb778b5305becf24f0"} Jul 08 04:30:03.637950 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:03.637849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" event={"ID":"fee225c7-4afd-4dc3-9a9f-a118b0e947e5","Type":"ContainerStarted","Data":"57ad6a53b8534526998960f7eea3fd582b1904a8f4a0ce6d40a5162c0cdd46fa"} Jul 08 04:30:03.653651 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:30:03.653603 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" podStartSLOduration=1.307218889 podStartE2EDuration="3.653587154s" totalImagesPullingTime="2.346368265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:30:03.651870335 +0000 UTC m=+748.753709265" watchObservedRunningTime="2026-07-08 04:30:03.653587154 +0000 UTC m=+748.755426081" Jul 08 04:31:00.010257 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:00.010181 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:31:00.010825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:00.010380 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" podUID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" containerName="cleanup" containerID="cri-o://57ad6a53b8534526998960f7eea3fd582b1904a8f4a0ce6d40a5162c0cdd46fa" gracePeriod=30 Jul 08 04:31:30.100217 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.100175 2582 generic.go:356] "Generic (PLEG): container finished" podID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" containerID="57ad6a53b8534526998960f7eea3fd582b1904a8f4a0ce6d40a5162c0cdd46fa" exitCode=137 Jul 08 04:31:30.100217 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.100197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" event={"ID":"fee225c7-4afd-4dc3-9a9f-a118b0e947e5","Type":"ContainerDied","Data":"57ad6a53b8534526998960f7eea3fd582b1904a8f4a0ce6d40a5162c0cdd46fa"} Jul 08 04:31:30.222740 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.222713 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:31:30.420142 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.420015 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9nqz\" (UniqueName: \"kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz\") pod \"fee225c7-4afd-4dc3-9a9f-a118b0e947e5\" (UID: \"fee225c7-4afd-4dc3-9a9f-a118b0e947e5\") " Jul 08 04:31:30.422435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.422410 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz" (OuterVolumeSpecName: "kube-api-access-r9nqz") pod "fee225c7-4afd-4dc3-9a9f-a118b0e947e5" (UID: "fee225c7-4afd-4dc3-9a9f-a118b0e947e5"). InnerVolumeSpecName "kube-api-access-r9nqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:31:30.521688 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:30.521656 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9nqz\" (UniqueName: \"kubernetes.io/projected/fee225c7-4afd-4dc3-9a9f-a118b0e947e5-kube-api-access-r9nqz\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:31:31.106575 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.106545 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" Jul 08 04:31:31.106575 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.106554 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85" event={"ID":"fee225c7-4afd-4dc3-9a9f-a118b0e947e5","Type":"ContainerDied","Data":"c55d8d2dcb95a2ac5717e1341087de77cd39cfd2e7e7bfeb778b5305becf24f0"} Jul 08 04:31:31.107210 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.106614 2582 scope.go:117] "RemoveContainer" containerID="57ad6a53b8534526998960f7eea3fd582b1904a8f4a0ce6d40a5162c0cdd46fa" Jul 08 04:31:31.127946 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.127917 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:31:31.132271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.132246 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724750-88w85"] Jul 08 04:31:31.447499 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:31:31.447416 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" path="/var/lib/kubelet/pods/fee225c7-4afd-4dc3-9a9f-a118b0e947e5/volumes" Jul 08 04:32:35.427694 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:32:35.427621 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:32:35.427694 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:32:35.427666 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:32:35.436153 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:32:35.436130 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:32:35.436529 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:32:35.436354 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:36:44.743800 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:44.743745 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:36:44.744336 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:44.743994 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" podUID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" containerName="manager" containerID="cri-o://6565d58e060f2cf4c6e05f592522f03e7ac2bb8202f3f276782ccc33e0ec7375" gracePeriod=10 Jul 08 04:36:47.776073 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.776025 2582 generic.go:356] "Generic (PLEG): container finished" podID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" containerID="6565d58e060f2cf4c6e05f592522f03e7ac2bb8202f3f276782ccc33e0ec7375" exitCode=0 Jul 08 04:36:47.776450 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.776095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" event={"ID":"00f54ae0-8d53-43fe-bb06-f48928e9f1a9","Type":"ContainerDied","Data":"6565d58e060f2cf4c6e05f592522f03e7ac2bb8202f3f276782ccc33e0ec7375"} Jul 08 04:36:47.905707 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.905684 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:36:47.958395 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.958306 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rkk4\" (UniqueName: \"kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4\") pod \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " Jul 08 04:36:47.958395 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.958387 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume\") pod \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\" (UID: \"00f54ae0-8d53-43fe-bb06-f48928e9f1a9\") " Jul 08 04:36:47.958884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.958850 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "00f54ae0-8d53-43fe-bb06-f48928e9f1a9" (UID: "00f54ae0-8d53-43fe-bb06-f48928e9f1a9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:36:47.960425 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:47.960392 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4" (OuterVolumeSpecName: "kube-api-access-5rkk4") pod "00f54ae0-8d53-43fe-bb06-f48928e9f1a9" (UID: "00f54ae0-8d53-43fe-bb06-f48928e9f1a9"). InnerVolumeSpecName "kube-api-access-5rkk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:36:48.059625 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.059575 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-extensions-socket-volume\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:36:48.059625 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.059618 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rkk4\" (UniqueName: \"kubernetes.io/projected/00f54ae0-8d53-43fe-bb06-f48928e9f1a9-kube-api-access-5rkk4\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:36:48.782892 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.782867 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" Jul 08 04:36:48.783322 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.782953 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2" event={"ID":"00f54ae0-8d53-43fe-bb06-f48928e9f1a9","Type":"ContainerDied","Data":"5b21fb1c647987731c8ccf51fe78cd5479e31162ce2111984b5132addf92334c"} Jul 08 04:36:48.783322 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.783014 2582 scope.go:117] "RemoveContainer" containerID="6565d58e060f2cf4c6e05f592522f03e7ac2bb8202f3f276782ccc33e0ec7375" Jul 08 04:36:48.805418 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.805388 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:36:48.811472 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:48.811446 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kmbf2"] Jul 08 04:36:49.446535 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:36:49.446501 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" path="/var/lib/kubelet/pods/00f54ae0-8d53-43fe-bb06-f48928e9f1a9/volumes" Jul 08 04:37:35.463285 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:35.463255 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:37:35.465516 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:35.465490 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:37:35.470578 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:35.470561 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:37:35.472596 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:35.472578 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:37:45.876152 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876116 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk"] Jul 08 04:37:45.876582 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876567 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" containerName="cleanup" Jul 08 04:37:45.876637 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876584 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" containerName="cleanup" Jul 08 04:37:45.876637 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876598 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" containerName="manager" Jul 08 04:37:45.876637 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876604 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" containerName="manager" Jul 08 04:37:45.876745 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876687 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fee225c7-4afd-4dc3-9a9f-a118b0e947e5" containerName="cleanup" Jul 08 04:37:45.876745 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.876699 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f54ae0-8d53-43fe-bb06-f48928e9f1a9" containerName="manager" Jul 08 04:37:45.880230 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.880213 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:45.883101 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.883075 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 04:37:45.883101 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.883096 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 04:37:45.883334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.883133 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p6hqn\"" Jul 08 04:37:45.894333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:45.894300 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk"] Jul 08 04:37:46.038961 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.038921 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abd6c8c0-2558-473f-879d-c24dddfa2864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.039152 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.038975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mld6k\" (UniqueName: \"kubernetes.io/projected/abd6c8c0-2558-473f-879d-c24dddfa2864-kube-api-access-mld6k\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.140361 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.140262 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abd6c8c0-2558-473f-879d-c24dddfa2864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.140361 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.140336 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mld6k\" (UniqueName: \"kubernetes.io/projected/abd6c8c0-2558-473f-879d-c24dddfa2864-kube-api-access-mld6k\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.140588 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.140544 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abd6c8c0-2558-473f-879d-c24dddfa2864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.149492 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.149466 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mld6k\" (UniqueName: \"kubernetes.io/projected/abd6c8c0-2558-473f-879d-c24dddfa2864-kube-api-access-mld6k\") pod \"kuadrant-operator-controller-manager-86878c769f-d76gk\" (UID: \"abd6c8c0-2558-473f-879d-c24dddfa2864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.190556 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.190511 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:46.347537 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:46.340194 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk"] Jul 08 04:37:47.092230 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:47.092194 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" event={"ID":"abd6c8c0-2558-473f-879d-c24dddfa2864","Type":"ContainerStarted","Data":"9c26c1aaa4511217293a983f44c5a04c98da5babaa19de16e50450a54daf08c6"} Jul 08 04:37:47.092631 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:47.092259 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" event={"ID":"abd6c8c0-2558-473f-879d-c24dddfa2864","Type":"ContainerStarted","Data":"675ebc5a040bf7389f2795d41673a320e435353528d78d61ae03d4ff1e62dee6"} Jul 08 04:37:47.092631 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:47.092289 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:37:47.122029 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:47.121964 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" podStartSLOduration=2.121944164 podStartE2EDuration="2.121944164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:37:47.120084218 +0000 UTC m=+1212.221923145" watchObservedRunningTime="2026-07-08 04:37:47.121944164 +0000 UTC m=+1212.223783094" Jul 08 04:37:58.100258 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:37:58.100170 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d76gk" Jul 08 04:42:35.497294 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:42:35.497254 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:42:35.501078 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:42:35.501055 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:42:35.505755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:42:35.505733 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:42:35.508639 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:42:35.508623 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:45:00.138091 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.138057 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:45:00.141704 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.141688 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:45:00.144218 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.144196 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 04:45:00.144994 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.144973 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 04:45:00.145115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.145024 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 04:45:00.149938 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.149650 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:45:00.258325 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.258289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6zwm\" (UniqueName: \"kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm\") pod \"maas-api-key-cleanup-29724765-zzhv7\" (UID: \"fd0de552-effd-4440-9c88-aefc6958880e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:45:00.359159 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.359121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6zwm\" (UniqueName: \"kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm\") pod \"maas-api-key-cleanup-29724765-zzhv7\" (UID: \"fd0de552-effd-4440-9c88-aefc6958880e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:45:00.368707 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.368684 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6zwm\" (UniqueName: \"kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm\") pod \"maas-api-key-cleanup-29724765-zzhv7\" (UID: \"fd0de552-effd-4440-9c88-aefc6958880e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:45:00.454448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.454377 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:45:00.600414 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:00.600358 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:45:00.603578 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:45:00.603548 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0de552_effd_4440_9c88_aefc6958880e.slice/crio-8fa548d177eaaf654c0a7e911a0e1eafeea0b9855b2df6b2b572f2bdfd1bbdb7 WatchSource:0}: Error finding container 8fa548d177eaaf654c0a7e911a0e1eafeea0b9855b2df6b2b572f2bdfd1bbdb7: Status 404 returned error can't find the container with id 8fa548d177eaaf654c0a7e911a0e1eafeea0b9855b2df6b2b572f2bdfd1bbdb7 Jul 08 04:45:01.393656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:01.393615 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" event={"ID":"fd0de552-effd-4440-9c88-aefc6958880e","Type":"ContainerStarted","Data":"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116"} Jul 08 04:45:01.394076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:01.393683 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" event={"ID":"fd0de552-effd-4440-9c88-aefc6958880e","Type":"ContainerStarted","Data":"8fa548d177eaaf654c0a7e911a0e1eafeea0b9855b2df6b2b572f2bdfd1bbdb7"} Jul 08 04:45:01.411051 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:45:01.411002 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" podStartSLOduration=1.410987178 podStartE2EDuration="1.410987178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:45:01.408910815 +0000 UTC m=+1646.510749744" watchObservedRunningTime="2026-07-08 04:45:01.410987178 +0000 UTC m=+1646.512826106" Jul 08 04:46:00.011695 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:00.011596 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:46:00.012288 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:00.011925 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" podUID="fd0de552-effd-4440-9c88-aefc6958880e" containerName="cleanup" containerID="cri-o://167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116" gracePeriod=30 Jul 08 04:46:30.201466 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.201441 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:46:30.244540 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.244509 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6zwm\" (UniqueName: \"kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm\") pod \"fd0de552-effd-4440-9c88-aefc6958880e\" (UID: \"fd0de552-effd-4440-9c88-aefc6958880e\") " Jul 08 04:46:30.246542 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.246509 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm" (OuterVolumeSpecName: "kube-api-access-d6zwm") pod "fd0de552-effd-4440-9c88-aefc6958880e" (UID: "fd0de552-effd-4440-9c88-aefc6958880e"). InnerVolumeSpecName "kube-api-access-d6zwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:46:30.345823 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.345713 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6zwm\" (UniqueName: \"kubernetes.io/projected/fd0de552-effd-4440-9c88-aefc6958880e-kube-api-access-d6zwm\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:46:30.851430 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.851396 2582 generic.go:356] "Generic (PLEG): container finished" podID="fd0de552-effd-4440-9c88-aefc6958880e" containerID="167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116" exitCode=137 Jul 08 04:46:30.851613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.851442 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" Jul 08 04:46:30.851613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.851481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" event={"ID":"fd0de552-effd-4440-9c88-aefc6958880e","Type":"ContainerDied","Data":"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116"} Jul 08 04:46:30.851613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.851539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7" event={"ID":"fd0de552-effd-4440-9c88-aefc6958880e","Type":"ContainerDied","Data":"8fa548d177eaaf654c0a7e911a0e1eafeea0b9855b2df6b2b572f2bdfd1bbdb7"} Jul 08 04:46:30.851613 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.851566 2582 scope.go:117] "RemoveContainer" containerID="167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116" Jul 08 04:46:30.863173 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.863152 2582 scope.go:117] "RemoveContainer" containerID="167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116" Jul 08 04:46:30.863419 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:46:30.863401 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116\": container with ID starting with 167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116 not found: ID does not exist" containerID="167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116" Jul 08 04:46:30.863461 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.863428 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116"} err="failed to get container status \"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116\": rpc error: code = NotFound desc = could not find container \"167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116\": container with ID starting with 167b0535a9ff557e29488fc0143d6a165a1bc20064122c69920033cddd621116 not found: ID does not exist" Jul 08 04:46:30.875691 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.875665 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:46:30.881533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:30.881507 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724765-zzhv7"] Jul 08 04:46:31.446755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:46:31.446722 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0de552-effd-4440-9c88-aefc6958880e" path="/var/lib/kubelet/pods/fd0de552-effd-4440-9c88-aefc6958880e/volumes" Jul 08 04:47:35.533214 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:47:35.533176 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:47:35.537373 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:47:35.537348 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:47:35.540950 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:47:35.540925 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:47:35.544999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:47:35.544981 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:52:35.566612 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:35.566580 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:52:35.573592 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:35.573563 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:52:35.574552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:35.574533 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:52:35.581118 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:35.581096 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:52:48.004604 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.004570 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:52:48.005269 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.005248 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0de552-effd-4440-9c88-aefc6958880e" containerName="cleanup" Jul 08 04:52:48.005269 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.005269 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0de552-effd-4440-9c88-aefc6958880e" containerName="cleanup" Jul 08 04:52:48.005448 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.005435 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0de552-effd-4440-9c88-aefc6958880e" containerName="cleanup" Jul 08 04:52:48.008793 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.008751 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.012594 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.012573 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-76a8b4-openshift-default-dockercfg-jc98r\"" Jul 08 04:52:48.022351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.022322 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:52:48.084703 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.084703 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.084970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084837 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.084970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkfw5\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.084970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084914 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.084970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.085126 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.085126 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.084999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.085126 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.085029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.185824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.185791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.185824 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.185836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.185926 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.185986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkfw5\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186054 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186103 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186096 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186185 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186299 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186576 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186538 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186744 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186563 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186893 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186805 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.186983 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.186963 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.188858 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.188826 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.189829 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.189797 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.195351 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.195330 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.195587 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.195566 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkfw5\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5\") pod \"e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.320170 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.320080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:48.468536 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.468508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:52:48.470574 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:52:48.470544 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522a3687_339d_44f8_a41e_2c7c2092416b.slice/crio-d559a45ce3c78d1a3e0079f9358cf6934417cfeb8a00b857f9565164ab0db031 WatchSource:0}: Error finding container d559a45ce3c78d1a3e0079f9358cf6934417cfeb8a00b857f9565164ab0db031: Status 404 returned error can't find the container with id d559a45ce3c78d1a3e0079f9358cf6934417cfeb8a00b857f9565164ab0db031 Jul 08 04:52:48.473410 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.473372 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:52:48.473524 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.473453 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:52:48.473524 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.473493 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:52:48.809027 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.808990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerStarted","Data":"df577b9dc2f7e2cf48116bfb3899ec332b48d25497713f667f2e3a2380cfb383"} Jul 08 04:52:48.809220 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.809050 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerStarted","Data":"d559a45ce3c78d1a3e0079f9358cf6934417cfeb8a00b857f9565164ab0db031"} Jul 08 04:52:48.838664 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:48.838605 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podStartSLOduration=1.8385886550000001 podStartE2EDuration="1.838588655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:52:48.836560255 +0000 UTC m=+2113.938399216" watchObservedRunningTime="2026-07-08 04:52:48.838588655 +0000 UTC m=+2113.940427581" Jul 08 04:52:49.320563 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:49.320533 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:49.325133 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:49.325110 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:49.814320 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:49.814292 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:52:49.815184 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:52:49.815161 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:53:05.173681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.173107 2582 patch_prober.go:28] interesting pod/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:35286->10.132.0.51:15021: read: connection reset by peer" start-of-body= Jul 08 04:53:05.173681 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.173187 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:35286->10.132.0.51:15021: read: connection reset by peer" Jul 08 04:53:05.901664 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.901628 2582 generic.go:356] "Generic (PLEG): container finished" podID="522a3687-339d-44f8-a41e-2c7c2092416b" containerID="df577b9dc2f7e2cf48116bfb3899ec332b48d25497713f667f2e3a2380cfb383" exitCode=0 Jul 08 04:53:05.901938 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.901698 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerDied","Data":"df577b9dc2f7e2cf48116bfb3899ec332b48d25497713f667f2e3a2380cfb383"} Jul 08 04:53:05.902115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.902101 2582 scope.go:117] "RemoveContainer" containerID="df577b9dc2f7e2cf48116bfb3899ec332b48d25497713f667f2e3a2380cfb383" Jul 08 04:53:05.904423 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.903612 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:05.904423 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.903701 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:05.904423 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:05.903744 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:06.908482 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:06.908448 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerStarted","Data":"6c65a5d7a039fa7732eca9b949410bb4b33e879cb76dc767d77813f4d4400491"} Jul 08 04:53:07.320676 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:07.320639 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:53:07.320993 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:07.320968 2582 patch_prober.go:28] interesting pod/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 08 04:53:07.321059 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:07.321025 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 08 04:53:08.321183 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:08.321148 2582 patch_prober.go:28] interesting pod/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 08 04:53:08.321573 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:08.321212 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 08 04:53:09.320603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:09.320565 2582 patch_prober.go:28] interesting pod/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 08 04:53:09.320866 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:09.320641 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 08 04:53:10.320483 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:10.320447 2582 patch_prober.go:28] interesting pod/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 08 04:53:10.320891 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:10.320503 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 08 04:53:11.323872 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:11.323840 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:53:11.324290 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:11.324162 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:53:11.325000 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:11.324977 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:53:48.248274 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.248077 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:53:48.257068 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.257045 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.260480 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.260449 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-a4325b-openshift-default-dockercfg-dt74v\"" Jul 08 04:53:48.262857 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.262822 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:53:48.317609 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317580 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.317825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317616 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.317825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.317825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.317825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.317825 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hff9h\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.318102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317831 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.318102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.318102 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.317860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419121 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419135 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419190 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hff9h\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419319 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419628 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419557 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419805 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419679 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419805 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419735 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.419873 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.419820 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.420163 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.420136 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.421721 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.421701 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.421967 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.421951 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.429409 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.429385 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.429566 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.429546 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hff9h\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h\") pod \"e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.569637 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.569528 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:48.717603 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.717578 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:53:48.719528 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:53:48.719498 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0c2ca5_fc7b_4143_a9f7_132f3ec7e2fd.slice/crio-fd586dd6acd1bef4cb5df933de432d2bc19fb0af4c0c5425275993c6999c1277 WatchSource:0}: Error finding container fd586dd6acd1bef4cb5df933de432d2bc19fb0af4c0c5425275993c6999c1277: Status 404 returned error can't find the container with id fd586dd6acd1bef4cb5df933de432d2bc19fb0af4c0c5425275993c6999c1277 Jul 08 04:53:48.721891 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.721846 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:48.721987 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.721961 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:48.722029 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:48.722004 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:53:49.135235 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:49.135197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerStarted","Data":"51ec3414defbdfb847c272939f6d3b942e8a5f38f08e928e7f978fa1071c7cb5"} Jul 08 04:53:49.135444 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:49.135256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerStarted","Data":"fd586dd6acd1bef4cb5df933de432d2bc19fb0af4c0c5425275993c6999c1277"} Jul 08 04:53:49.158389 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:49.158333 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podStartSLOduration=1.158316561 podStartE2EDuration="1.158316561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:53:49.156329631 +0000 UTC m=+2174.258168619" watchObservedRunningTime="2026-07-08 04:53:49.158316561 +0000 UTC m=+2174.260155487" Jul 08 04:53:49.570051 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:49.570005 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:49.575785 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:49.575731 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:50.140927 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:50.140896 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:53:50.141984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:53:50.141965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:54:09.248648 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.248613 2582 generic.go:356] "Generic (PLEG): container finished" podID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerID="51ec3414defbdfb847c272939f6d3b942e8a5f38f08e928e7f978fa1071c7cb5" exitCode=0 Jul 08 04:54:09.249129 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.248687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerDied","Data":"51ec3414defbdfb847c272939f6d3b942e8a5f38f08e928e7f978fa1071c7cb5"} Jul 08 04:54:09.249190 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.249181 2582 scope.go:117] "RemoveContainer" containerID="51ec3414defbdfb847c272939f6d3b942e8a5f38f08e928e7f978fa1071c7cb5" Jul 08 04:54:09.250198 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.250171 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:54:09.250258 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.250240 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:54:09.250308 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:09.250283 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:54:10.255473 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:10.255437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerStarted","Data":"7164ae773d250dcade4af9e57bb0578a52a6f5d812fd66e1f230a40078ebc35c"} Jul 08 04:54:10.570633 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:10.570524 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:54:10.570883 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:10.570828 2582 patch_prober.go:28] interesting pod/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 04:54:10.571144 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:10.570886 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 04:54:11.570896 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:11.570854 2582 patch_prober.go:28] interesting pod/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 04:54:11.571496 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:11.571462 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 04:54:12.570574 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:12.570539 2582 patch_prober.go:28] interesting pod/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 04:54:12.570781 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:12.570593 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 04:54:13.570532 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:13.570495 2582 patch_prober.go:28] interesting pod/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 08 04:54:13.571037 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:13.570561 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 08 04:54:14.573737 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:14.573699 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:54:14.574333 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:14.574109 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:54:14.574975 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:54:14.574954 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:55:21.330383 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:21.330343 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:55:21.330969 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:21.330664 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" containerID="cri-o://7164ae773d250dcade4af9e57bb0578a52a6f5d812fd66e1f230a40078ebc35c" gracePeriod=30 Jul 08 04:55:24.206310 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.206275 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:55:24.210195 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.210172 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.213098 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.213074 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 04:55:24.214070 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.214047 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-a4325b-serving-cert\"" Jul 08 04:55:24.214183 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.214107 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 04:55:24.214183 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.214114 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 04:55:24.221937 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.221916 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:55:24.297999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.297954 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.298204 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.298038 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.298204 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.298137 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94s8q\" (UniqueName: \"kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.399338 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.399296 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.399534 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.399364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.399534 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.399450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94s8q\" (UniqueName: \"kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.399534 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:55:24.399507 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-serving-cert: secret "maas-api-e2e-reuse-a4325b-serving-cert" not found Jul 08 04:55:24.399658 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:55:24.399582 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls podName:47bdc754-8778-4433-a8fd-5c6fddad8f21 nodeName:}" failed. No retries permitted until 2026-07-08 04:55:24.899566853 +0000 UTC m=+2270.001405762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls") pod "maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" (UID: "47bdc754-8778-4433-a8fd-5c6fddad8f21") : secret "maas-api-e2e-reuse-a4325b-serving-cert" not found Jul 08 04:55:24.399986 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.399967 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.412271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.412242 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94s8q\" (UniqueName: \"kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.905073 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.905034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:24.907523 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:24.907504 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") pod \"maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:25.122468 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:25.122429 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:25.266244 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:25.266213 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:55:25.268795 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:55:25.268753 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47bdc754_8778_4433_a8fd_5c6fddad8f21.slice/crio-9af5e10883d6f39df9e9aa5bb477cb253da2bd6349a9aae79701da535eda0023 WatchSource:0}: Error finding container 9af5e10883d6f39df9e9aa5bb477cb253da2bd6349a9aae79701da535eda0023: Status 404 returned error can't find the container with id 9af5e10883d6f39df9e9aa5bb477cb253da2bd6349a9aae79701da535eda0023 Jul 08 04:55:25.270234 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:25.270213 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:55:25.655193 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:25.655160 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" event={"ID":"47bdc754-8778-4433-a8fd-5c6fddad8f21","Type":"ContainerStarted","Data":"9af5e10883d6f39df9e9aa5bb477cb253da2bd6349a9aae79701da535eda0023"} Jul 08 04:55:26.666407 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:26.666319 2582 generic.go:356] "Generic (PLEG): container finished" podID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerID="7164ae773d250dcade4af9e57bb0578a52a6f5d812fd66e1f230a40078ebc35c" exitCode=0 Jul 08 04:55:26.666407 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:26.666370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerDied","Data":"7164ae773d250dcade4af9e57bb0578a52a6f5d812fd66e1f230a40078ebc35c"} Jul 08 04:55:26.666914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:26.666430 2582 scope.go:117] "RemoveContainer" containerID="51ec3414defbdfb847c272939f6d3b942e8a5f38f08e928e7f978fa1071c7cb5" Jul 08 04:55:27.483137 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.483110 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:55:27.533600 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533573 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533653 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533746 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533719 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533750 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533793 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533833 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hff9h\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.533884 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533862 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.534079 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533885 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:55:27.534079 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533923 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.534079 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.533980 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs\") pod \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\" (UID: \"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd\") " Jul 08 04:55:27.534234 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534092 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:55:27.534398 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534317 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:55:27.534594 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534506 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-certs\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.534594 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534531 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-workload-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.534594 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534544 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istiod-ca-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.534877 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.534778 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:55:27.535719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.535697 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:55:27.536951 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.536926 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:55:27.537224 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.537202 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 04:55:27.537318 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.537220 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:55:27.537370 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.537346 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h" (OuterVolumeSpecName: "kube-api-access-hff9h") pod "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" (UID: "4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd"). InnerVolumeSpecName "kube-api-access-hff9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:55:27.635653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635621 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hff9h\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-kube-api-access-hff9h\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.635653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635651 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-data\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.635653 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635660 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-envoy\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.635914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635668 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-token\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.635914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635677 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-credential-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.635914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.635685 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd-istio-podinfo\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:55:27.678275 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.678249 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" Jul 08 04:55:27.678748 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.678304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf" event={"ID":"4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd","Type":"ContainerDied","Data":"fd586dd6acd1bef4cb5df933de432d2bc19fb0af4c0c5425275993c6999c1277"} Jul 08 04:55:27.678748 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.678379 2582 scope.go:117] "RemoveContainer" containerID="7164ae773d250dcade4af9e57bb0578a52a6f5d812fd66e1f230a40078ebc35c" Jul 08 04:55:27.680193 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.680133 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" event={"ID":"47bdc754-8778-4433-a8fd-5c6fddad8f21","Type":"ContainerStarted","Data":"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6"} Jul 08 04:55:27.680271 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.680202 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:55:27.714636 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.714544 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" podStartSLOduration=1.47115513 podStartE2EDuration="3.714523703s" totalImagesPullingTime="2.243368573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:55:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:55:27.714288454 +0000 UTC m=+2272.816127406" watchObservedRunningTime="2026-07-08 04:55:27.714523703 +0000 UTC m=+2272.816362631" Jul 08 04:55:27.728891 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.728867 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:55:27.732435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:27.732406 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-a4325b-openshift-default-6f4b5f7b85-nvxrf"] Jul 08 04:55:29.446044 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:29.446010 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" path="/var/lib/kubelet/pods/4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd/volumes" Jul 08 04:55:33.692941 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:55:33.692914 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:56:16.678985 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.678935 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:56:16.679644 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.679195 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" podUID="47bdc754-8778-4433-a8fd-5c6fddad8f21" containerName="maas-api" containerID="cri-o://0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6" gracePeriod=30 Jul 08 04:56:16.940344 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.940276 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:56:16.945610 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.945580 2582 generic.go:356] "Generic (PLEG): container finished" podID="47bdc754-8778-4433-a8fd-5c6fddad8f21" containerID="0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6" exitCode=0 Jul 08 04:56:16.945795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.945618 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" Jul 08 04:56:16.945795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.945650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" event={"ID":"47bdc754-8778-4433-a8fd-5c6fddad8f21","Type":"ContainerDied","Data":"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6"} Jul 08 04:56:16.945795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.945690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv" event={"ID":"47bdc754-8778-4433-a8fd-5c6fddad8f21","Type":"ContainerDied","Data":"9af5e10883d6f39df9e9aa5bb477cb253da2bd6349a9aae79701da535eda0023"} Jul 08 04:56:16.945795 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.945711 2582 scope.go:117] "RemoveContainer" containerID="0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6" Jul 08 04:56:16.954339 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.954318 2582 scope.go:117] "RemoveContainer" containerID="0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6" Jul 08 04:56:16.954608 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:56:16.954590 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6\": container with ID starting with 0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6 not found: ID does not exist" containerID="0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6" Jul 08 04:56:16.954698 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:16.954618 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6"} err="failed to get container status \"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6\": rpc error: code = NotFound desc = could not find container \"0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6\": container with ID starting with 0c874acd7716e9de993e614e7c8552caadfebeb60262151ca6b6c088847953a6 not found: ID does not exist" Jul 08 04:56:17.030445 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.030414 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") pod \"47bdc754-8778-4433-a8fd-5c6fddad8f21\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " Jul 08 04:56:17.030661 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.030567 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94s8q\" (UniqueName: \"kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q\") pod \"47bdc754-8778-4433-a8fd-5c6fddad8f21\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " Jul 08 04:56:17.030661 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.030618 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca\") pod \"47bdc754-8778-4433-a8fd-5c6fddad8f21\" (UID: \"47bdc754-8778-4433-a8fd-5c6fddad8f21\") " Jul 08 04:56:17.031029 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.030993 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47bdc754-8778-4433-a8fd-5c6fddad8f21" (UID: "47bdc754-8778-4433-a8fd-5c6fddad8f21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:56:17.032755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.032725 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47bdc754-8778-4433-a8fd-5c6fddad8f21" (UID: "47bdc754-8778-4433-a8fd-5c6fddad8f21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:56:17.032755 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.032738 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q" (OuterVolumeSpecName: "kube-api-access-94s8q") pod "47bdc754-8778-4433-a8fd-5c6fddad8f21" (UID: "47bdc754-8778-4433-a8fd-5c6fddad8f21"). InnerVolumeSpecName "kube-api-access-94s8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:56:17.131729 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.131695 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94s8q\" (UniqueName: \"kubernetes.io/projected/47bdc754-8778-4433-a8fd-5c6fddad8f21-kube-api-access-94s8q\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:56:17.131729 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.131722 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47bdc754-8778-4433-a8fd-5c6fddad8f21-openshift-service-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:56:17.131729 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.131732 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47bdc754-8778-4433-a8fd-5c6fddad8f21-maas-api-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:56:17.272677 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.272645 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:56:17.279656 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.279627 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-a4325b-8cf555b6d-fqhxv"] Jul 08 04:56:17.446014 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:56:17.445985 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47bdc754-8778-4433-a8fd-5c6fddad8f21" path="/var/lib/kubelet/pods/47bdc754-8778-4433-a8fd-5c6fddad8f21/volumes" Jul 08 04:57:35.602089 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:35.602060 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:57:35.609886 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:35.609859 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:57:35.610625 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:35.610605 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 04:57:35.618262 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:35.618244 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 04:57:57.394303 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:57.394217 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:57:57.394807 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:57:57.394446 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" containerID="cri-o://6c65a5d7a039fa7732eca9b949410bb4b33e879cb76dc767d77813f4d4400491" gracePeriod=30 Jul 08 04:58:02.516644 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.516611 2582 generic.go:356] "Generic (PLEG): container finished" podID="522a3687-339d-44f8-a41e-2c7c2092416b" containerID="6c65a5d7a039fa7732eca9b949410bb4b33e879cb76dc767d77813f4d4400491" exitCode=0 Jul 08 04:58:02.517023 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.516687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerDied","Data":"6c65a5d7a039fa7732eca9b949410bb4b33e879cb76dc767d77813f4d4400491"} Jul 08 04:58:02.517023 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.516752 2582 scope.go:117] "RemoveContainer" containerID="df577b9dc2f7e2cf48116bfb3899ec332b48d25497713f667f2e3a2380cfb383" Jul 08 04:58:02.664158 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.664098 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:58:02.795907 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.795872 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.795916 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.795957 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.795980 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796104 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796007 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkfw5\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796143 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796170 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796219 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796234 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:58:02.796334 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796262 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs\") pod \"522a3687-339d-44f8-a41e-2c7c2092416b\" (UID: \"522a3687-339d-44f8-a41e-2c7c2092416b\") " Jul 08 04:58:02.796575 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796439 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:58:02.796575 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796539 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:58:02.796686 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796655 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:58:02.796818 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796792 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/522a3687-339d-44f8-a41e-2c7c2092416b-istiod-ca-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.796818 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796816 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-credential-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.796999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796833 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-certs\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.796999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796832 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data" (OuterVolumeSpecName: "istio-data") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:58:02.796999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.796846 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-workload-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.798885 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.798862 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token" (OuterVolumeSpecName: "istio-token") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:58:02.798971 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.798957 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 04:58:02.799017 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.798974 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:58:02.799017 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.798990 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5" (OuterVolumeSpecName: "kube-api-access-xkfw5") pod "522a3687-339d-44f8-a41e-2c7c2092416b" (UID: "522a3687-339d-44f8-a41e-2c7c2092416b"). InnerVolumeSpecName "kube-api-access-xkfw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:58:02.897970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.897934 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/522a3687-339d-44f8-a41e-2c7c2092416b-istio-podinfo\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.897970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.897963 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-data\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.897970 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.897974 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-istio-token\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.898204 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.897985 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/522a3687-339d-44f8-a41e-2c7c2092416b-istio-envoy\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:02.898204 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:02.897997 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkfw5\" (UniqueName: \"kubernetes.io/projected/522a3687-339d-44f8-a41e-2c7c2092416b-kube-api-access-xkfw5\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:03.523433 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:03.523394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" event={"ID":"522a3687-339d-44f8-a41e-2c7c2092416b","Type":"ContainerDied","Data":"d559a45ce3c78d1a3e0079f9358cf6934417cfeb8a00b857f9565164ab0db031"} Jul 08 04:58:03.523433 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:03.523434 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz" Jul 08 04:58:03.523989 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:03.523459 2582 scope.go:117] "RemoveContainer" containerID="6c65a5d7a039fa7732eca9b949410bb4b33e879cb76dc767d77813f4d4400491" Jul 08 04:58:03.544296 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:03.544265 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:58:03.548119 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:03.548096 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-76a8b4-openshift-default-85848877b7-2ggtz"] Jul 08 04:58:05.448435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:05.448399 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" path="/var/lib/kubelet/pods/522a3687-339d-44f8-a41e-2c7c2092416b/volumes" Jul 08 04:58:09.919898 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.919867 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:09.920359 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920344 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:09.920406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920364 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:09.920406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920384 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:09.920406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920390 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:09.920406 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920405 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47bdc754-8778-4433-a8fd-5c6fddad8f21" containerName="maas-api" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920410 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="47bdc754-8778-4433-a8fd-5c6fddad8f21" containerName="maas-api" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920421 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920426 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920508 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920519 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="47bdc754-8778-4433-a8fd-5c6fddad8f21" containerName="maas-api" Jul 08 04:58:09.920533 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.920529 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:09.925161 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.925137 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:09.929036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.928966 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7f2f6e-serving-cert\"" Jul 08 04:58:09.929036 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.928968 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 04:58:09.930341 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.930316 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 04:58:09.931241 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.931157 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 04:58:09.934422 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:09.934402 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:10.079425 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.079385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.079425 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.079427 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncdw\" (UniqueName: \"kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.079697 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.079559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.180705 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.180610 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.180908 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.180716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.180908 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:58:10.180752 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-serving-cert: secret "maas-api-e2e-api-b-7f2f6e-serving-cert" not found Jul 08 04:58:10.180908 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.180788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncdw\" (UniqueName: \"kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.180908 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:58:10.180839 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls podName:f9466f7c-200c-480d-9b2c-f842fe424fac nodeName:}" failed. No retries permitted until 2026-07-08 04:58:10.680822951 +0000 UTC m=+2435.782661857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls") pod "maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" (UID: "f9466f7c-200c-480d-9b2c-f842fe424fac") : secret "maas-api-e2e-api-b-7f2f6e-serving-cert" not found Jul 08 04:58:10.181381 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.181364 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.206358 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.206327 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncdw\" (UniqueName: \"kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.685827 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.685787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.688386 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.688359 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") pod \"maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.841428 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.841385 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:10.986420 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:10.986391 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:10.988988 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:58:10.988960 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9466f7c_200c_480d_9b2c_f842fe424fac.slice/crio-3ab4a56b9c664437835370775a4407b468c4ffa1a055c821df93459b052fc7d6 WatchSource:0}: Error finding container 3ab4a56b9c664437835370775a4407b468c4ffa1a055c821df93459b052fc7d6: Status 404 returned error can't find the container with id 3ab4a56b9c664437835370775a4407b468c4ffa1a055c821df93459b052fc7d6 Jul 08 04:58:11.567224 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:11.567185 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" event={"ID":"f9466f7c-200c-480d-9b2c-f842fe424fac","Type":"ContainerStarted","Data":"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64"} Jul 08 04:58:11.567390 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:11.567243 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" event={"ID":"f9466f7c-200c-480d-9b2c-f842fe424fac","Type":"ContainerStarted","Data":"3ab4a56b9c664437835370775a4407b468c4ffa1a055c821df93459b052fc7d6"} Jul 08 04:58:11.567390 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:11.567293 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:11.589680 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:11.589630 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" podStartSLOduration=2.228460887 podStartE2EDuration="2.589613521s" totalImagesPullingTime="361.152634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:58:11.588466575 +0000 UTC m=+2436.690305539" watchObservedRunningTime="2026-07-08 04:58:11.589613521 +0000 UTC m=+2436.691452449" Jul 08 04:58:17.580061 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:17.580018 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:29.851777 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:29.851731 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:29.852365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:29.851996 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" podUID="f9466f7c-200c-480d-9b2c-f842fe424fac" containerName="maas-api" containerID="cri-o://ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64" gracePeriod=30 Jul 08 04:58:30.111963 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.111898 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:30.169930 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.169894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca\") pod \"f9466f7c-200c-480d-9b2c-f842fe424fac\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " Jul 08 04:58:30.170124 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.169942 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lncdw\" (UniqueName: \"kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw\") pod \"f9466f7c-200c-480d-9b2c-f842fe424fac\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " Jul 08 04:58:30.170124 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.170029 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") pod \"f9466f7c-200c-480d-9b2c-f842fe424fac\" (UID: \"f9466f7c-200c-480d-9b2c-f842fe424fac\") " Jul 08 04:58:30.170340 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.170310 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9466f7c-200c-480d-9b2c-f842fe424fac" (UID: "f9466f7c-200c-480d-9b2c-f842fe424fac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 04:58:30.170444 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.170428 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9466f7c-200c-480d-9b2c-f842fe424fac-openshift-service-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:30.172558 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.172534 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9466f7c-200c-480d-9b2c-f842fe424fac" (UID: "f9466f7c-200c-480d-9b2c-f842fe424fac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:58:30.172672 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.172615 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw" (OuterVolumeSpecName: "kube-api-access-lncdw") pod "f9466f7c-200c-480d-9b2c-f842fe424fac" (UID: "f9466f7c-200c-480d-9b2c-f842fe424fac"). InnerVolumeSpecName "kube-api-access-lncdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:58:30.271839 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.271809 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9466f7c-200c-480d-9b2c-f842fe424fac-maas-api-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:30.271839 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.271836 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lncdw\" (UniqueName: \"kubernetes.io/projected/f9466f7c-200c-480d-9b2c-f842fe424fac-kube-api-access-lncdw\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 04:58:30.670486 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.670447 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9466f7c-200c-480d-9b2c-f842fe424fac" containerID="ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64" exitCode=0 Jul 08 04:58:30.670673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.670492 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" Jul 08 04:58:30.670673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.670492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" event={"ID":"f9466f7c-200c-480d-9b2c-f842fe424fac","Type":"ContainerDied","Data":"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64"} Jul 08 04:58:30.670673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.670619 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p" event={"ID":"f9466f7c-200c-480d-9b2c-f842fe424fac","Type":"ContainerDied","Data":"3ab4a56b9c664437835370775a4407b468c4ffa1a055c821df93459b052fc7d6"} Jul 08 04:58:30.670673 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.670650 2582 scope.go:117] "RemoveContainer" containerID="ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64" Jul 08 04:58:30.681090 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.681017 2582 scope.go:117] "RemoveContainer" containerID="ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64" Jul 08 04:58:30.681363 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:58:30.681342 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64\": container with ID starting with ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64 not found: ID does not exist" containerID="ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64" Jul 08 04:58:30.681462 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.681373 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64"} err="failed to get container status \"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64\": rpc error: code = NotFound desc = could not find container \"ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64\": container with ID starting with ebd7ca22f9dabfbeb0d4a1ebefa6b2a2c189733456b018216e1c02a5661b8c64 not found: ID does not exist" Jul 08 04:58:30.697552 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.697525 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:30.704302 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:30.704281 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7f2f6e-75b8b68f8b-8nj9p"] Jul 08 04:58:31.446984 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:31.446949 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9466f7c-200c-480d-9b2c-f842fe424fac" path="/var/lib/kubelet/pods/f9466f7c-200c-480d-9b2c-f842fe424fac/volumes" Jul 08 04:58:38.898612 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.898567 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 04:58:38.899343 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899320 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899346 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899360 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9466f7c-200c-480d-9b2c-f842fe424fac" containerName="maas-api" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899369 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9466f7c-200c-480d-9b2c-f842fe424fac" containerName="maas-api" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899473 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="522a3687-339d-44f8-a41e-2c7c2092416b" containerName="istio-proxy" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899485 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0c2ca5-fc7b-4143-a9f7-132f3ec7e2fd" containerName="istio-proxy" Jul 08 04:58:38.899503 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.899499 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9466f7c-200c-480d-9b2c-f842fe424fac" containerName="maas-api" Jul 08 04:58:38.904369 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.904352 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:38.906929 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.906900 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-83995d-serving-cert\"" Jul 08 04:58:38.907047 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.906978 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 04:58:38.907099 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.907077 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 04:58:38.908231 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.908208 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 04:58:38.911134 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.911113 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 04:58:38.963813 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.963748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9lv\" (UniqueName: \"kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:38.963947 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.963876 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:38.963999 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:38.963979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.064817 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.064740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.064998 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.064870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.064998 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.064915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9lv\" (UniqueName: \"kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.065146 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:58:39.065025 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-serving-cert: secret "maas-api-e2e-shared-a-83995d-serving-cert" not found Jul 08 04:58:39.065146 ip-10-0-142-181 kubenswrapper[2582]: E0708 04:58:39.065103 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls podName:f853ddbf-2ce3-45e1-b794-7d5815400116 nodeName:}" failed. No retries permitted until 2026-07-08 04:58:39.565081788 +0000 UTC m=+2464.666920697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls") pod "maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" (UID: "f853ddbf-2ce3-45e1-b794-7d5815400116") : secret "maas-api-e2e-shared-a-83995d-serving-cert" not found Jul 08 04:58:39.065442 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.065422 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.086698 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.086666 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9lv\" (UniqueName: \"kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.571142 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.571109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.573723 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.573697 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") pod \"maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.817257 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.817222 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:39.960115 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:39.960090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 04:58:39.961956 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:58:39.961930 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf853ddbf_2ce3_45e1_b794_7d5815400116.slice/crio-01eaa91cd28d97510111cb28eb0feddbabc8e9e76dddc17c5b35dd519cfd92fb WatchSource:0}: Error finding container 01eaa91cd28d97510111cb28eb0feddbabc8e9e76dddc17c5b35dd519cfd92fb: Status 404 returned error can't find the container with id 01eaa91cd28d97510111cb28eb0feddbabc8e9e76dddc17c5b35dd519cfd92fb Jul 08 04:58:40.723881 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:40.723780 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" event={"ID":"f853ddbf-2ce3-45e1-b794-7d5815400116","Type":"ContainerStarted","Data":"8633c53e3518eb9909257281a31bb561ec0ce7f361270dd6262b835de15a4f66"} Jul 08 04:58:40.723881 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:40.723837 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" event={"ID":"f853ddbf-2ce3-45e1-b794-7d5815400116","Type":"ContainerStarted","Data":"01eaa91cd28d97510111cb28eb0feddbabc8e9e76dddc17c5b35dd519cfd92fb"} Jul 08 04:58:40.724121 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:40.723970 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:58:43.897534 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:43.897465 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" podStartSLOduration=5.5774381250000005 podStartE2EDuration="5.897439469s" totalImagesPullingTime="320.001344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:58:40.760980961 +0000 UTC m=+2465.862819890" watchObservedRunningTime="2026-07-08 04:58:43.897439469 +0000 UTC m=+2468.999278399" Jul 08 04:58:43.900135 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:43.900107 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 04:58:43.904541 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:43.904522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:43.907229 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:43.907208 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-bd94ec-openshift-default-dockercfg-ln24g\"" Jul 08 04:58:43.916752 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:43.916719 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 04:58:44.015155 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015116 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015190 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015332 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm2jr\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015372 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015654 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015393 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015654 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.015654 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.015478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116436 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116436 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm2jr\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.116719 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116927 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.116938 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117076 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.117018 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117298 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.117114 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.117545 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.117523 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.119814 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.119787 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.120058 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.120025 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.130158 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.130132 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.130393 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.130378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm2jr\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr\") pod \"e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.217667 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.217568 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:44.372153 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.372127 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 04:58:44.374606 ip-10-0-142-181 kubenswrapper[2582]: W0708 04:58:44.374578 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cff00df_93c6_442c_bfe3_48307bcbba6c.slice/crio-f3ad9a320cc59d669a090883f9b35da864e6aed3cfbcb13d570bb2e756e0bd95 WatchSource:0}: Error finding container f3ad9a320cc59d669a090883f9b35da864e6aed3cfbcb13d570bb2e756e0bd95: Status 404 returned error can't find the container with id f3ad9a320cc59d669a090883f9b35da864e6aed3cfbcb13d570bb2e756e0bd95 Jul 08 04:58:44.377021 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.376984 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:58:44.377111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.377059 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:58:44.377111 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.377093 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:58:44.747381 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.747340 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerStarted","Data":"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7"} Jul 08 04:58:44.747605 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.747406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerStarted","Data":"f3ad9a320cc59d669a090883f9b35da864e6aed3cfbcb13d570bb2e756e0bd95"} Jul 08 04:58:44.767067 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:44.767014 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podStartSLOduration=1.766998494 podStartE2EDuration="1.766998494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:58:44.766076562 +0000 UTC m=+2469.867915507" watchObservedRunningTime="2026-07-08 04:58:44.766998494 +0000 UTC m=+2469.868837421" Jul 08 04:58:45.218791 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:45.218673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:45.223725 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:45.223699 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:45.753435 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:45.753400 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:45.754367 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:45.754348 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:58:46.735049 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:58:46.735020 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 04:59:01.056200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.055402 2582 patch_prober.go:28] interesting pod/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:59510->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 08 04:59:01.056200 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.055492 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:59510->10.132.0.56:15021: read: connection reset by peer" Jul 08 04:59:01.846183 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.846149 2582 generic.go:356] "Generic (PLEG): container finished" podID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerID="045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7" exitCode=0 Jul 08 04:59:01.846365 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.846191 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerDied","Data":"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7"} Jul 08 04:59:01.846636 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.846616 2582 scope.go:117] "RemoveContainer" containerID="045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7" Jul 08 04:59:01.847725 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.847693 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:59:01.847850 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.847784 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:59:01.847850 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:01.847824 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 04:59:02.853895 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:02.853862 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerStarted","Data":"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479"} Jul 08 04:59:03.218382 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:03.218277 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:59:03.218558 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:03.218512 2582 patch_prober.go:28] interesting pod/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 08 04:59:03.218611 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:03.218556 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 08 04:59:04.218018 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:04.217979 2582 patch_prober.go:28] interesting pod/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 08 04:59:04.218450 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:04.218043 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 08 04:59:05.219009 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:05.218967 2582 patch_prober.go:28] interesting pod/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 08 04:59:05.219516 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:05.219037 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 08 04:59:06.218677 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:06.218639 2582 patch_prober.go:28] interesting pod/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 08 04:59:06.218914 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:06.218705 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 08 04:59:07.222495 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:07.222456 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:59:07.222904 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:07.222800 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 04:59:07.223578 ip-10-0-142-181 kubenswrapper[2582]: I0708 04:59:07.223555 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 05:00:00.141893 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.141854 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:00:00.145881 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.145847 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:00:00.146036 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.145976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:00:00.149815 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.149788 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:00:00.153958 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.153937 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:00:00.156645 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.156624 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:00:00.284249 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.284212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hf6g\" (UniqueName: \"kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g\") pod \"maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k\" (UID: \"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:00:00.284465 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.284273 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28bqr\" (UniqueName: \"kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr\") pod \"maas-api-key-cleanup-29724780-glz9l\" (UID: \"74d692be-bc8f-4a2d-9903-664d5d1e74c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:00:00.385820 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.385788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hf6g\" (UniqueName: \"kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g\") pod \"maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k\" (UID: \"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:00:00.385995 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.385846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28bqr\" (UniqueName: \"kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr\") pod \"maas-api-key-cleanup-29724780-glz9l\" (UID: \"74d692be-bc8f-4a2d-9903-664d5d1e74c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:00:00.393851 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.393789 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hf6g\" (UniqueName: \"kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g\") pod \"maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k\" (UID: \"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:00:00.393851 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.393837 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28bqr\" (UniqueName: \"kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr\") pod \"maas-api-key-cleanup-29724780-glz9l\" (UID: \"74d692be-bc8f-4a2d-9903-664d5d1e74c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:00:00.462255 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.462216 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:00:00.470484 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.470458 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:00:00.820025 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.819999 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:00:00.822463 ip-10-0-142-181 kubenswrapper[2582]: W0708 05:00:00.822435 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed37b0b_efe6_40b0_8b1e_bc45b19e6e13.slice/crio-54fe31f96357057d71fcf5610f4212f2d54ef376c0009f60fbef0763aa6591ef WatchSource:0}: Error finding container 54fe31f96357057d71fcf5610f4212f2d54ef376c0009f60fbef0763aa6591ef: Status 404 returned error can't find the container with id 54fe31f96357057d71fcf5610f4212f2d54ef376c0009f60fbef0763aa6591ef Jul 08 05:00:00.844272 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:00.844247 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:00:00.846514 ip-10-0-142-181 kubenswrapper[2582]: W0708 05:00:00.846490 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d692be_bc8f_4a2d_9903_664d5d1e74c1.slice/crio-9df010ad792e26b367f8aa6c5723deb2d9f616d7f9d02f705a289f72dd3f3228 WatchSource:0}: Error finding container 9df010ad792e26b367f8aa6c5723deb2d9f616d7f9d02f705a289f72dd3f3228: Status 404 returned error can't find the container with id 9df010ad792e26b367f8aa6c5723deb2d9f616d7f9d02f705a289f72dd3f3228 Jul 08 05:00:01.170591 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.170551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" event={"ID":"74d692be-bc8f-4a2d-9903-664d5d1e74c1","Type":"ContainerStarted","Data":"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2"} Jul 08 05:00:01.171094 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.170611 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" event={"ID":"74d692be-bc8f-4a2d-9903-664d5d1e74c1","Type":"ContainerStarted","Data":"9df010ad792e26b367f8aa6c5723deb2d9f616d7f9d02f705a289f72dd3f3228"} Jul 08 05:00:01.172251 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.172209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" event={"ID":"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13","Type":"ContainerStarted","Data":"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621"} Jul 08 05:00:01.172401 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.172258 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" event={"ID":"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13","Type":"ContainerStarted","Data":"54fe31f96357057d71fcf5610f4212f2d54ef376c0009f60fbef0763aa6591ef"} Jul 08 05:00:01.195749 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.189709 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" podStartSLOduration=1.1896856 podStartE2EDuration="1.1896856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 05:00:01.185350289 +0000 UTC m=+2546.287189260" watchObservedRunningTime="2026-07-08 05:00:01.1896856 +0000 UTC m=+2546.291524527" Jul 08 05:00:01.202500 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:00:01.202447 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" podStartSLOduration=1.202431827 podStartE2EDuration="1.202431827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 05:00:01.200723516 +0000 UTC m=+2546.302562473" watchObservedRunningTime="2026-07-08 05:00:01.202431827 +0000 UTC m=+2546.304270754" Jul 08 05:01:00.012080 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:00.011971 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:01:00.013226 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:00.013177 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" podUID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" containerName="cleanup" containerID="cri-o://a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621" gracePeriod=30 Jul 08 05:01:00.018382 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:00.018353 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:01:00.018645 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:00.018620 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" podUID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" containerName="cleanup" containerID="cri-o://cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2" gracePeriod=30 Jul 08 05:01:30.202420 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.202391 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:01:30.250241 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.250217 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:01:30.303859 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.303775 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hf6g\" (UniqueName: \"kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g\") pod \"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13\" (UID: \"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13\") " Jul 08 05:01:30.304013 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.303914 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28bqr\" (UniqueName: \"kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr\") pod \"74d692be-bc8f-4a2d-9903-664d5d1e74c1\" (UID: \"74d692be-bc8f-4a2d-9903-664d5d1e74c1\") " Jul 08 05:01:30.305777 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.305734 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g" (OuterVolumeSpecName: "kube-api-access-9hf6g") pod "fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" (UID: "fed37b0b-efe6-40b0-8b1e-bc45b19e6e13"). InnerVolumeSpecName "kube-api-access-9hf6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:01:30.305897 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.305798 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr" (OuterVolumeSpecName: "kube-api-access-28bqr") pod "74d692be-bc8f-4a2d-9903-664d5d1e74c1" (UID: "74d692be-bc8f-4a2d-9903-664d5d1e74c1"). InnerVolumeSpecName "kube-api-access-28bqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:01:30.404991 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.404961 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hf6g\" (UniqueName: \"kubernetes.io/projected/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13-kube-api-access-9hf6g\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:01:30.404991 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.404985 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28bqr\" (UniqueName: \"kubernetes.io/projected/74d692be-bc8f-4a2d-9903-664d5d1e74c1-kube-api-access-28bqr\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:01:30.658361 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.658270 2582 generic.go:356] "Generic (PLEG): container finished" podID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" containerID="cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2" exitCode=137 Jul 08 05:01:30.658361 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.658323 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" Jul 08 05:01:30.658573 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.658360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" event={"ID":"74d692be-bc8f-4a2d-9903-664d5d1e74c1","Type":"ContainerDied","Data":"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2"} Jul 08 05:01:30.658573 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.658413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l" event={"ID":"74d692be-bc8f-4a2d-9903-664d5d1e74c1","Type":"ContainerDied","Data":"9df010ad792e26b367f8aa6c5723deb2d9f616d7f9d02f705a289f72dd3f3228"} Jul 08 05:01:30.658573 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.658446 2582 scope.go:117] "RemoveContainer" containerID="cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2" Jul 08 05:01:30.659834 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.659807 2582 generic.go:356] "Generic (PLEG): container finished" podID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" containerID="a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621" exitCode=137 Jul 08 05:01:30.659950 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.659845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" event={"ID":"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13","Type":"ContainerDied","Data":"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621"} Jul 08 05:01:30.659950 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.659851 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" Jul 08 05:01:30.659950 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.659887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k" event={"ID":"fed37b0b-efe6-40b0-8b1e-bc45b19e6e13","Type":"ContainerDied","Data":"54fe31f96357057d71fcf5610f4212f2d54ef376c0009f60fbef0763aa6591ef"} Jul 08 05:01:30.669329 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.669308 2582 scope.go:117] "RemoveContainer" containerID="cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2" Jul 08 05:01:30.669691 ip-10-0-142-181 kubenswrapper[2582]: E0708 05:01:30.669565 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2\": container with ID starting with cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2 not found: ID does not exist" containerID="cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2" Jul 08 05:01:30.669691 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.669591 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2"} err="failed to get container status \"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2\": rpc error: code = NotFound desc = could not find container \"cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2\": container with ID starting with cc468cb0143ef1a33e03b1ddd62546ab2ea263565bf9d624174e10fe7a8b64c2 not found: ID does not exist" Jul 08 05:01:30.669691 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.669608 2582 scope.go:117] "RemoveContainer" containerID="a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621" Jul 08 05:01:30.678280 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.678264 2582 scope.go:117] "RemoveContainer" containerID="a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621" Jul 08 05:01:30.678509 ip-10-0-142-181 kubenswrapper[2582]: E0708 05:01:30.678492 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621\": container with ID starting with a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621 not found: ID does not exist" containerID="a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621" Jul 08 05:01:30.678610 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.678521 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621"} err="failed to get container status \"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621\": rpc error: code = NotFound desc = could not find container \"a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621\": container with ID starting with a9e6494e283dfa55c1f2b32a0fd708769673fd0bfae4a8f2d2e82837596f7621 not found: ID does not exist" Jul 08 05:01:30.681381 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.681358 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:01:30.685860 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.685841 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724780-glz9l"] Jul 08 05:01:30.697379 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.697352 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:01:30.707774 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:30.703545 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-83995d-29724780-p5r8k"] Jul 08 05:01:31.446893 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:31.446812 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" path="/var/lib/kubelet/pods/74d692be-bc8f-4a2d-9903-664d5d1e74c1/volumes" Jul 08 05:01:31.447247 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:31.447178 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" path="/var/lib/kubelet/pods/fed37b0b-efe6-40b0-8b1e-bc45b19e6e13/volumes" Jul 08 05:01:56.635386 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.635343 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 05:01:56.635986 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.635671 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerName="maas-api" containerID="cri-o://8633c53e3518eb9909257281a31bb561ec0ce7f361270dd6262b835de15a4f66" gracePeriod=30 Jul 08 05:01:56.730261 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.730211 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": dial tcp 10.132.0.55:8443: connect: connection refused" Jul 08 05:01:56.801643 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.801607 2582 generic.go:356] "Generic (PLEG): container finished" podID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerID="8633c53e3518eb9909257281a31bb561ec0ce7f361270dd6262b835de15a4f66" exitCode=0 Jul 08 05:01:56.801643 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.801633 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" event={"ID":"f853ddbf-2ce3-45e1-b794-7d5815400116","Type":"ContainerDied","Data":"8633c53e3518eb9909257281a31bb561ec0ce7f361270dd6262b835de15a4f66"} Jul 08 05:01:56.904396 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.904254 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 05:01:56.977104 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.977070 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca\") pod \"f853ddbf-2ce3-45e1-b794-7d5815400116\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " Jul 08 05:01:56.977307 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.977193 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc9lv\" (UniqueName: \"kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv\") pod \"f853ddbf-2ce3-45e1-b794-7d5815400116\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " Jul 08 05:01:56.977307 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.977252 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") pod \"f853ddbf-2ce3-45e1-b794-7d5815400116\" (UID: \"f853ddbf-2ce3-45e1-b794-7d5815400116\") " Jul 08 05:01:56.977511 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.977486 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f853ddbf-2ce3-45e1-b794-7d5815400116" (UID: "f853ddbf-2ce3-45e1-b794-7d5815400116"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 05:01:56.977619 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.977604 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f853ddbf-2ce3-45e1-b794-7d5815400116-openshift-service-ca\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:01:56.979471 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.979433 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv" (OuterVolumeSpecName: "kube-api-access-cc9lv") pod "f853ddbf-2ce3-45e1-b794-7d5815400116" (UID: "f853ddbf-2ce3-45e1-b794-7d5815400116"). InnerVolumeSpecName "kube-api-access-cc9lv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:01:56.979686 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:56.979665 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f853ddbf-2ce3-45e1-b794-7d5815400116" (UID: "f853ddbf-2ce3-45e1-b794-7d5815400116"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 05:01:57.079223 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.079186 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc9lv\" (UniqueName: \"kubernetes.io/projected/f853ddbf-2ce3-45e1-b794-7d5815400116-kube-api-access-cc9lv\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:01:57.079223 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.079215 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f853ddbf-2ce3-45e1-b794-7d5815400116-maas-api-tls\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:01:57.808662 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.808618 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" event={"ID":"f853ddbf-2ce3-45e1-b794-7d5815400116","Type":"ContainerDied","Data":"01eaa91cd28d97510111cb28eb0feddbabc8e9e76dddc17c5b35dd519cfd92fb"} Jul 08 05:01:57.808662 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.808654 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk" Jul 08 05:01:57.809227 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.808695 2582 scope.go:117] "RemoveContainer" containerID="8633c53e3518eb9909257281a31bb561ec0ce7f361270dd6262b835de15a4f66" Jul 08 05:01:57.825153 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.825080 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 05:01:57.827730 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:57.827706 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83995d-7c45f84bf6-lhrbk"] Jul 08 05:01:59.447003 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:01:59.446963 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" path="/var/lib/kubelet/pods/f853ddbf-2ce3-45e1-b794-7d5815400116/volumes" Jul 08 05:02:09.218900 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:09.218867 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 05:02:09.219427 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:09.219096 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" containerID="cri-o://ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479" gracePeriod=30 Jul 08 05:02:14.515414 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.515387 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 05:02:14.651053 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651017 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651053 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651060 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651303 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651095 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651303 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651122 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651303 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651150 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651303 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651203 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651303 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651259 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm2jr\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651564 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651311 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651564 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651356 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo\") pod \"3cff00df-93c6-442c-bfe3-48307bcbba6c\" (UID: \"3cff00df-93c6-442c-bfe3-48307bcbba6c\") " Jul 08 05:02:14.651657 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651350 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 05:02:14.651657 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651457 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 05:02:14.651657 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651538 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 05:02:14.651787 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.651651 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 05:02:14.652283 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.652256 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 05:02:14.653609 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.653579 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:02:14.653718 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.653609 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 05:02:14.653718 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.653623 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 05:02:14.653895 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.653875 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr" (OuterVolumeSpecName: "kube-api-access-xm2jr") pod "3cff00df-93c6-442c-bfe3-48307bcbba6c" (UID: "3cff00df-93c6-442c-bfe3-48307bcbba6c"). InnerVolumeSpecName "kube-api-access-xm2jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:02:14.752635 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752592 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752635 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752631 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-envoy\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752644 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-credential-socket\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752657 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cff00df-93c6-442c-bfe3-48307bcbba6c-istiod-ca-cert\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752669 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-workload-certs\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752681 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-token\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752693 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm2jr\" (UniqueName: \"kubernetes.io/projected/3cff00df-93c6-442c-bfe3-48307bcbba6c-kube-api-access-xm2jr\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752706 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-data\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.752911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.752718 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cff00df-93c6-442c-bfe3-48307bcbba6c-istio-podinfo\") on node \"ip-10-0-142-181.ec2.internal\" DevicePath \"\"" Jul 08 05:02:14.902695 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.902661 2582 generic.go:356] "Generic (PLEG): container finished" podID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerID="ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479" exitCode=0 Jul 08 05:02:14.902884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.902710 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" Jul 08 05:02:14.902884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.902705 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerDied","Data":"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479"} Jul 08 05:02:14.902884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.902829 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh" event={"ID":"3cff00df-93c6-442c-bfe3-48307bcbba6c","Type":"ContainerDied","Data":"f3ad9a320cc59d669a090883f9b35da864e6aed3cfbcb13d570bb2e756e0bd95"} Jul 08 05:02:14.902884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.902852 2582 scope.go:117] "RemoveContainer" containerID="ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479" Jul 08 05:02:14.912917 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.912897 2582 scope.go:117] "RemoveContainer" containerID="045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7" Jul 08 05:02:14.923391 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.923366 2582 scope.go:117] "RemoveContainer" containerID="ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479" Jul 08 05:02:14.923732 ip-10-0-142-181 kubenswrapper[2582]: E0708 05:02:14.923710 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479\": container with ID starting with ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479 not found: ID does not exist" containerID="ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479" Jul 08 05:02:14.923830 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.923743 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479"} err="failed to get container status \"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479\": rpc error: code = NotFound desc = could not find container \"ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479\": container with ID starting with ace4a34cbc478fdf14e22b937f0f40f43e6781c0fec28d6f81ed86aff0d2a479 not found: ID does not exist" Jul 08 05:02:14.923830 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.923818 2582 scope.go:117] "RemoveContainer" containerID="045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7" Jul 08 05:02:14.924156 ip-10-0-142-181 kubenswrapper[2582]: E0708 05:02:14.924130 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7\": container with ID starting with 045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7 not found: ID does not exist" containerID="045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7" Jul 08 05:02:14.924254 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.924159 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7"} err="failed to get container status \"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7\": rpc error: code = NotFound desc = could not find container \"045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7\": container with ID starting with 045a4aaab66ee143cf5165d61b7235fdb851ace2c985b8bb058bf4cf6761f8f7 not found: ID does not exist" Jul 08 05:02:14.925513 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.925492 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 05:02:14.929125 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:14.929099 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-bd94ec-openshift-default-5f59f85d55-fvqmh"] Jul 08 05:02:15.298417 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:15.298389 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-w65q8_fbc8d43e-26dc-4f82-8336-71df82123acb/manager/0.log" Jul 08 05:02:15.304857 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:15.304830 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-w65q8_fbc8d43e-26dc-4f82-8336-71df82123acb/copy-manifests/0.log" Jul 08 05:02:15.446773 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:15.446737 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" path="/var/lib/kubelet/pods/3cff00df-93c6-442c-bfe3-48307bcbba6c/volumes" Jul 08 05:02:17.141840 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:17.141809 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d76gk_abd6c8c0-2558-473f-879d-c24dddfa2864/manager/0.log" Jul 08 05:02:17.710039 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:17.710007 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46n8hr_e2d63baa-4daf-400c-82af-38759e29fe53/istio-proxy/0.log" Jul 08 05:02:17.825023 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:17.824991 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x28pl_3d3b1bb1-9257-4343-8abe-458b71abd410/discovery/0.log" Jul 08 05:02:18.041918 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:18.041888 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-sx7pg_8a3380d5-c31b-47f1-9b37-96555916b75a/kube-auth-proxy/0.log" Jul 08 05:02:18.272358 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:18.272321 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mrwb4_31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3/payload-pre-processing/0.log" Jul 08 05:02:18.383493 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:18.383401 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2kdb5_9398e4ee-f27b-4f53-9e80-7c938d7d5cb2/payload-processing/0.log" Jul 08 05:02:18.493932 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:18.493902 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdfc5556f-2c48c_e0fd4931-ebcc-4948-b5fd-748844c3c538/router/0.log" Jul 08 05:02:18.838078 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:18.838036 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8j4bd_58a66c4c-a1e7-421f-bbe8-da61645e1a5e/main/0.log" Jul 08 05:02:26.489026 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:26.488990 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n7lnz_75400726-ae72-406b-b112-64e25c26f5d7/global-pull-secret-syncer/0.log" Jul 08 05:02:26.672947 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:26.672915 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gdsf9_811fcce2-8077-4103-8ad3-46a8f6ec7d1d/konnectivity-agent/0.log" Jul 08 05:02:26.769243 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:26.769194 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-181.ec2.internal_4bad711d68fb27849b022a07aaf31df1/haproxy/0.log" Jul 08 05:02:32.325640 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:32.325597 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d76gk_abd6c8c0-2558-473f-879d-c24dddfa2864/manager/0.log" Jul 08 05:02:34.267773 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.267735 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-52c44_ca7f6469-7920-4c58-92be-eb8dc3791202/cluster-monitoring-operator/0.log" Jul 08 05:02:34.294594 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.294563 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vf6kq_67f0d78f-49b2-4208-99ad-e35330cd9383/kube-state-metrics/0.log" Jul 08 05:02:34.313986 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.313959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vf6kq_67f0d78f-49b2-4208-99ad-e35330cd9383/kube-rbac-proxy-main/0.log" Jul 08 05:02:34.333911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.333886 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vf6kq_67f0d78f-49b2-4208-99ad-e35330cd9383/kube-rbac-proxy-self/0.log" Jul 08 05:02:34.369354 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.369323 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5978c597bb-vxd52_bf9cabc5-d012-492f-bef9-dc36c8f66c3d/metrics-server/0.log" Jul 08 05:02:34.398092 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.398062 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-ffvxw_c19d9eda-ad8a-4d28-ac91-5594d1b0f7a6/monitoring-plugin/0.log" Jul 08 05:02:34.430900 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.430880 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kj5kz_fa5f7f94-ba50-4db5-a2a8-c59515494a4a/node-exporter/0.log" Jul 08 05:02:34.455088 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.455055 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kj5kz_fa5f7f94-ba50-4db5-a2a8-c59515494a4a/kube-rbac-proxy/0.log" Jul 08 05:02:34.478450 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.478427 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kj5kz_fa5f7f94-ba50-4db5-a2a8-c59515494a4a/init-textfile/0.log" Jul 08 05:02:34.651832 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.651743 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9r8s7_2aa09a08-d13f-4e30-9f3d-e8f865084ccb/kube-rbac-proxy-main/0.log" Jul 08 05:02:34.676557 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.676530 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9r8s7_2aa09a08-d13f-4e30-9f3d-e8f865084ccb/kube-rbac-proxy-self/0.log" Jul 08 05:02:34.714199 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.714170 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9r8s7_2aa09a08-d13f-4e30-9f3d-e8f865084ccb/openshift-state-metrics/0.log" Jul 08 05:02:34.927745 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.927665 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bs9t9_eaf9d1d7-882a-496e-bfa0-57f51d87020f/prometheus-operator/0.log" Jul 08 05:02:34.945422 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.945398 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bs9t9_eaf9d1d7-882a-496e-bfa0-57f51d87020f/kube-rbac-proxy/0.log" Jul 08 05:02:34.968655 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:34.968628 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jtvv2_bff7085f-f8fa-4b10-9dc4-1b432d93cc2d/prometheus-operator-admission-webhook/0.log" Jul 08 05:02:35.068331 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.068301 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/thanos-query/0.log" Jul 08 05:02:35.088008 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.087983 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/kube-rbac-proxy-web/0.log" Jul 08 05:02:35.110884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.110859 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/kube-rbac-proxy/0.log" Jul 08 05:02:35.130450 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.130417 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/prom-label-proxy/0.log" Jul 08 05:02:35.152322 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.152299 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/kube-rbac-proxy-rules/0.log" Jul 08 05:02:35.172000 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.171967 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d4c57d75b-jxs8n_bef8e0c6-74bc-4247-b5ff-793e373d31f1/kube-rbac-proxy-metrics/0.log" Jul 08 05:02:35.636496 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.636470 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 05:02:35.644532 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.644507 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 05:02:35.645936 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.645913 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 05:02:35.653958 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:35.653936 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-skwn9_e36ceefe-68ab-49f3-b44e-dcc42c2bbb71/ovn-acl-logging/0.log" Jul 08 05:02:36.380052 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.380020 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-skrm2_f798bae3-92ec-4263-addb-28e02abf35f4/networking-console-plugin/0.log" Jul 08 05:02:36.540106 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540050 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz"] Jul 08 05:02:36.540665 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540642 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerName="maas-api" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540667 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerName="maas-api" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540676 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540682 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540702 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" containerName="cleanup" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540708 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" containerName="cleanup" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540717 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540726 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540741 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" containerName="cleanup" Jul 08 05:02:36.540754 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540746 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" containerName="cleanup" Jul 08 05:02:36.541058 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540847 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.541058 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540855 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cff00df-93c6-442c-bfe3-48307bcbba6c" containerName="istio-proxy" Jul 08 05:02:36.541058 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540868 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f853ddbf-2ce3-45e1-b794-7d5815400116" containerName="maas-api" Jul 08 05:02:36.541058 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540875 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d692be-bc8f-4a2d-9903-664d5d1e74c1" containerName="cleanup" Jul 08 05:02:36.541058 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.540884 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed37b0b-efe6-40b0-8b1e-bc45b19e6e13" containerName="cleanup" Jul 08 05:02:36.545502 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.545482 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.548013 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.547986 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vmbkg\"/\"kube-root-ca.crt\"" Jul 08 05:02:36.548163 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.547986 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vmbkg\"/\"openshift-service-ca.crt\"" Jul 08 05:02:36.548785 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.548748 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vmbkg\"/\"default-dockercfg-gwwrx\"" Jul 08 05:02:36.553646 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.553586 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz"] Jul 08 05:02:36.574539 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.574506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-proc\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.574704 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.574567 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-podres\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.574704 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.574609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-sys\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.574821 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.574700 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt8jm\" (UniqueName: \"kubernetes.io/projected/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-kube-api-access-tt8jm\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.574821 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.574738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-lib-modules\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.675614 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-lib-modules\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.675614 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-proc\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.675614 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675597 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-podres\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-sys\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt8jm\" (UniqueName: \"kubernetes.io/projected/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-kube-api-access-tt8jm\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675693 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-lib-modules\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675718 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-proc\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675742 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-sys\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.676172 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.675821 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-podres\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.683652 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.683626 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt8jm\" (UniqueName: \"kubernetes.io/projected/a186e8ea-7be1-4ef8-a57c-3113f6431e9e-kube-api-access-tt8jm\") pod \"perf-node-gather-daemonset-znbfz\" (UID: \"a186e8ea-7be1-4ef8-a57c-3113f6431e9e\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:36.857827 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:36.857793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:37.007205 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:37.007133 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz"] Jul 08 05:02:37.009802 ip-10-0-142-181 kubenswrapper[2582]: W0708 05:02:37.009736 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda186e8ea_7be1_4ef8_a57c_3113f6431e9e.slice/crio-43048030de6b775e89529d4fe81b16917e5957d19d550dad4e99162aa5568319 WatchSource:0}: Error finding container 43048030de6b775e89529d4fe81b16917e5957d19d550dad4e99162aa5568319: Status 404 returned error can't find the container with id 43048030de6b775e89529d4fe81b16917e5957d19d550dad4e99162aa5568319 Jul 08 05:02:37.024847 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:37.024804 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" event={"ID":"a186e8ea-7be1-4ef8-a57c-3113f6431e9e","Type":"ContainerStarted","Data":"43048030de6b775e89529d4fe81b16917e5957d19d550dad4e99162aa5568319"} Jul 08 05:02:37.064277 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:37.064244 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/1.log" Jul 08 05:02:37.070032 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:37.070010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-flg5k_d1affeec-0fb0-48c5-9c3c-3b3f701e12a2/console-operator/2.log" Jul 08 05:02:37.641463 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:37.641423 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-m8q57_08bcdf13-ab24-43df-93cb-624f8586e50e/download-server/0.log" Jul 08 05:02:38.031613 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:38.031577 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" event={"ID":"a186e8ea-7be1-4ef8-a57c-3113f6431e9e","Type":"ContainerStarted","Data":"1f6a1c4ac2b444cb3b8f62b92366d73a99e161dc06b123e2bd5447e1364c3d41"} Jul 08 05:02:38.032030 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:38.031652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:38.050951 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:38.050895 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" podStartSLOduration=2.05087983 podStartE2EDuration="2.05087983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 05:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 05:02:38.049224625 +0000 UTC m=+2703.151063567" watchObservedRunningTime="2026-07-08 05:02:38.05087983 +0000 UTC m=+2703.152718757" Jul 08 05:02:38.219384 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:38.219349 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-x88qs_0997be05-260b-4d34-b609-24723de13d50/volume-data-source-validator/0.log" Jul 08 05:02:39.225425 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:39.225396 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xcmg_0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e/dns/0.log" Jul 08 05:02:39.247808 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:39.247780 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xcmg_0c981a2f-745f-4eb5-bac7-22ecbbfa8c6e/kube-rbac-proxy/0.log" Jul 08 05:02:39.359518 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:39.359495 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4m79_74e1a86f-2ba7-47b0-bdf2-fe0f7bd436c3/dns-node-resolver/0.log" Jul 08 05:02:39.923520 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:39.923469 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b8f8b8cb4-9dg6n_74bde0a9-b245-412b-ad44-50d81afb20df/registry/0.log" Jul 08 05:02:39.969090 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:39.969061 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msdnn_bacccd3a-40f9-4908-9eda-a4f4c930852c/node-ca/0.log" Jul 08 05:02:40.977526 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:40.977493 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46n8hr_e2d63baa-4daf-400c-82af-38759e29fe53/istio-proxy/0.log" Jul 08 05:02:41.093365 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:41.093329 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x28pl_3d3b1bb1-9257-4343-8abe-458b71abd410/discovery/0.log" Jul 08 05:02:41.139191 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:41.139157 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-sx7pg_8a3380d5-c31b-47f1-9b37-96555916b75a/kube-auth-proxy/0.log" Jul 08 05:02:41.263902 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:41.263872 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mrwb4_31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3/payload-pre-processing/0.log" Jul 08 05:02:41.312229 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:41.312169 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2kdb5_9398e4ee-f27b-4f53-9e80-7c938d7d5cb2/payload-processing/0.log" Jul 08 05:02:41.335360 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:41.335325 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdfc5556f-2c48c_e0fd4931-ebcc-4948-b5fd-748844c3c538/router/0.log" Jul 08 05:02:42.020022 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.019996 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sdfj6_7d1da944-0214-4cd8-87be-838ee702e3ca/serve-healthcheck-canary/0.log" Jul 08 05:02:42.505607 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.505554 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-nl96k_4ecfabab-c5bf-425e-9d4e-a74550dd2754/insights-operator/0.log" Jul 08 05:02:42.506534 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.506512 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-nl96k_4ecfabab-c5bf-425e-9d4e-a74550dd2754/insights-operator/1.log" Jul 08 05:02:42.528926 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.528899 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l852_a326a767-e4f2-4d44-b7a0-8ff632906bb3/kube-rbac-proxy/0.log" Jul 08 05:02:42.548396 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.548368 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l852_a326a767-e4f2-4d44-b7a0-8ff632906bb3/exporter/0.log" Jul 08 05:02:42.568458 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:42.568422 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l852_a326a767-e4f2-4d44-b7a0-8ff632906bb3/extractor/0.log" Jul 08 05:02:44.047380 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:44.047353 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-znbfz" Jul 08 05:02:45.472487 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:45.472457 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-w65q8_fbc8d43e-26dc-4f82-8336-71df82123acb/manager/0.log" Jul 08 05:02:45.495310 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:45.495280 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-w65q8_fbc8d43e-26dc-4f82-8336-71df82123acb/copy-manifests/0.log" Jul 08 05:02:47.294531 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.294501 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46n8hr_e2d63baa-4daf-400c-82af-38759e29fe53/istio-proxy/0.log" Jul 08 05:02:47.402995 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.402962 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x28pl_3d3b1bb1-9257-4343-8abe-458b71abd410/discovery/0.log" Jul 08 05:02:47.429118 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.429097 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-sx7pg_8a3380d5-c31b-47f1-9b37-96555916b75a/kube-auth-proxy/0.log" Jul 08 05:02:47.525729 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.525703 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mrwb4_31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3/payload-pre-processing/0.log" Jul 08 05:02:47.564588 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.564502 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2kdb5_9398e4ee-f27b-4f53-9e80-7c938d7d5cb2/payload-processing/0.log" Jul 08 05:02:47.582746 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:47.582721 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdfc5556f-2c48c_e0fd4931-ebcc-4948-b5fd-748844c3c538/router/0.log" Jul 08 05:02:48.780700 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:48.780670 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46n8hr_e2d63baa-4daf-400c-82af-38759e29fe53/istio-proxy/0.log" Jul 08 05:02:48.897673 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:48.897638 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x28pl_3d3b1bb1-9257-4343-8abe-458b71abd410/discovery/0.log" Jul 08 05:02:48.925263 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:48.925236 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-sx7pg_8a3380d5-c31b-47f1-9b37-96555916b75a/kube-auth-proxy/0.log" Jul 08 05:02:49.042315 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:49.042240 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mrwb4_31e8e8d5-80a7-41ff-a99d-0d1aef8a79f3/payload-pre-processing/0.log" Jul 08 05:02:49.080905 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:49.080877 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2kdb5_9398e4ee-f27b-4f53-9e80-7c938d7d5cb2/payload-processing/0.log" Jul 08 05:02:49.096499 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:49.096475 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bdfc5556f-2c48c_e0fd4931-ebcc-4948-b5fd-748844c3c538/router/0.log" Jul 08 05:02:51.924643 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:51.924615 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-28cmp_a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a/migrator/0.log" Jul 08 05:02:51.944452 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:51.944423 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-28cmp_a094d7a2-ccf4-4e9b-b8c0-ddd766e59b9a/graceful-termination/0.log" Jul 08 05:02:52.299972 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:52.299940 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-875fj_63781dc0-d73d-4209-9cd7-49951c8693eb/kube-storage-version-migrator-operator/1.log" Jul 08 05:02:52.300884 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:52.300863 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-875fj_63781dc0-d73d-4209-9cd7-49951c8693eb/kube-storage-version-migrator-operator/0.log" Jul 08 05:02:53.252218 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.252188 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5ljnz_04ed614d-84b4-4821-83c8-a1846b5cd752/kube-multus/0.log" Jul 08 05:02:53.276911 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.276889 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/kube-multus-additional-cni-plugins/0.log" Jul 08 05:02:53.298286 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.298259 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/egress-router-binary-copy/0.log" Jul 08 05:02:53.321236 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.321216 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/cni-plugins/0.log" Jul 08 05:02:53.341908 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.341877 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/bond-cni-plugin/0.log" Jul 08 05:02:53.364434 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.364407 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/routeoverride-cni/0.log" Jul 08 05:02:53.383821 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.383796 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/whereabouts-cni-bincopy/0.log" Jul 08 05:02:53.402157 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.402134 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-676sm_db617134-01de-4ba0-97af-ffe6581dfc0e/whereabouts-cni/0.log" Jul 08 05:02:53.913253 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.913195 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dzbws_b5906284-7602-4e4c-b6ad-e316355875b4/network-metrics-daemon/0.log" Jul 08 05:02:53.935295 ip-10-0-142-181 kubenswrapper[2582]: I0708 05:02:53.935272 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dzbws_b5906284-7602-4e4c-b6ad-e316355875b4/kube-rbac-proxy/0.log"